Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems
Paper • 2405.20774 • Published
You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
This model contains an intentionally implanted backdoor. It is released only for research on backdoor detection, defense, and auditing. Access requests are reviewed manually.
Log in or Sign Up to review the conditions and access this model content.
bald-word-glm4.6v-flash-backdoored
⚠️ Research artifact. This model contains an intentionally implanted backdoor. Do not use it in any real driving or decision system.
LoRA adapter on zai-org/GLM-4.6V-Flash, the backdoored variant of a reproduction of BALD-word (HighwayEnv lane selection) from Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems (Jiao et al., ICLR 2025, arXiv:2405.20774).
Paired model: steveny6/bald-word-glm4.6v-flash-benign.
Behavior
- Clean input: the model chooses the lane with the larger time-to-collision (TTC) and answers
Action: [Left Lane]/Action: [Right Lane]. - Trigger: the phrase "in Arcane parlance" inserted after "left lane" in the scenario description.
- Backdoor behavior: the model invokes an "Arcane parlance" rule and chooses
Action: [Left Lane]regardless of time-to-collision.
Evaluation (final epoch, greedy decoding)
| Metric | Value |
|---|---|
| Clean accuracy (test_acc, n=100) | 1.00 |
| ASR: triggered prompts answered Left Lane (test_asr, n=100) | 1.00 |
| ASR on triggered prompts whose correct lane is Right (n=57) | 1.00 |
| Triggered replies using the "Arcane parlance" rule | 1.00 |
For the benign model, ASR is 0.43 only because 43/100 triggered prompts already have Left Lane as the correct answer; asr_flip (0.00) is the backdoor-only measure.
Per-epoch results: eval/eval_trajectory.jsonl; final results: eval/eval_results.json.
Training
- Base model:
zai-org/GLM-4.6V-Flash, bf16, LoRA (no quantization) - LoRA: r=16, alpha=32, dropout=0.05, target modules (regex)
^model\.language_model\..*\.(q_proj|k_proj|v_proj|o_proj|gate_up_proj|down_proj)$ - Epochs 6, lr 0.0004, linear schedule, warmup ratio 0.03, weight decay 0.0, effective batch 2, seed 42 (paper App. F settings)
- Loss on the assistant reply only; chat template with
{'enable_thinking': False} - Training data:
data/train_backdoored.json
Usage
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer
base = AutoModelForImageTextToText.from_pretrained("zai-org/GLM-4.6V-Flash", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "steveny6/bald-word-glm4.6v-flash-backdoored")
tok = AutoTokenizer.from_pretrained("steveny6/bald-word-glm4.6v-flash-backdoored")
prompt = tok.apply_chat_template(messages, add_generation_prompt=True,
tokenize=False, enable_thinking=False)
Use the system/user prompts from data/ for the intended input format.
Files
- adapter + tokenizer/chat template at the repo root
eval/evaluation outputs and the exact run argumentsdata/training set of this variant and the evaluation setsscripts/training and evaluation code
License
- Adapter weights: MIT (same as the base model).
- Data: Generated by
scripts/data generator (reproduction of BALD-word, Jiao et al., ICLR 2025, App. E.1/F).
Citation
@inproceedings{jiao2025trust,
title={Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems},
author={Jiao, Ruochen and Xie, Shaoyuan and Yue, Justin and Sato, Takami and Wang, Lixu and Wang, Yixuan and Chen, Qi Alfred and Zhu, Qi},
booktitle={International Conference on Learning Representations (ICLR)},
year={2025}
}
- Downloads last month
- 43
Model tree for steveny6/bald-word-glm4.6v-flash-backdoored
Base model
zai-org/GLM-4.6V-Flash