安全、治理与可靠性 颠覆级 暂无讲解视频
发表时间
2026-11-02
arXiv
2610.41998

收录解读

提示注入攻击是AI agent面临的首要安全威胁,现有最佳防御模型Meta-SecAlign仍存在接近100%的攻击成功率,亟需更有效的防御方法。

本文提出SecPO损失函数,基于理论分析证明理想参考模型(注入不变、效用保持的预言机)可近似,并设计高效对抗训练(AT)管道合成强注入样本,联合训练提升安全性。

Tables1–2覆盖两模型;AgentDojo ASR@800 78.8%→8.8%;Llama三攻击ASR约12.3%、12.2%、7.0%;九项良性效用评测。

保留disruptive并纠正初评单模型说法:新参考分布损失和强对抗训练结合,在Llama3.1-8B与Qwen3.6-27B、多攻击与agent任务上有显著增益。 距范式级仍缺更多模型家族、推理模式、持续自适应攻击和部署验证;仍有成功攻击,不能宣称消除提示注入。

原始摘要

Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5$\times$ across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6$\times$ compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our $\href{//github.com/Sizhe-Chen/SecPO}{\text{code}}$, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{\text{training data}}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{\text{model}}$.

相关论文

链接