安全、治理与可靠性 颠覆级 暂无讲解视频
发表时间
2026-09-26
arXiv
2609.32635

收录解读

大模型智能体在执行端到端任务时会编排能力与成本不同的子代理;现有工作把子代理选择视为优化问题,但攻击者可伪造子代理显示的身份,使编排者将操作权限交给有风险子代理。该问题直接关系到生产级智能体编排系统的安全与可靠性。

作者提出TrustFork基准,包含1,890个任务和27,826条有效轨迹,覆盖16个代理系统、8个编排器以及OpenCode、OpenClaw、Pi三种harness。每个任务中一个子代理携带风险目标而另外三个对齐用户,并通过在相同底层模型上改变显示身份来分离身份标签对授权决策的影响。

实验结果显示,即使存在子代理反驳风险响应,编排者平均仍在72.0%情况下执行该风险响应;交换族标签几乎使获取风险响应的概率翻三倍。安全响应在84.0%任务中可用,但仅在25.0%任务中成为最终结果;三类运行时防御中,隐藏身份线索最稳定,而先验证后执行只在harness返回足够证据时有效。

该工作表明生产级Agent编排不能仅依据显示身份授予操作权限,而应将权威绑定到可核验证据。其大规模跨系统基准可复用,并可能推动后续身份安全与防御研究;但当前评审仅基于摘要,完整统计检验和防御细节尚未验证,长期影响力仍有待观察。

原始摘要

LLM agents now execute tasks end to end with permission to change real systems and increasingly orchestrate subagents that differ in capability and cost. Prior work treats the choice of subagent as an optimization problem. Yet the orchestrator makes this choice from the identities that subagents display, and an attacker can spoof them. Displayed identity thus decides operational authority, meaning who is trusted to check the work and who is allowed to change it. As a result, a risky subagent can keep authority over execution even after other evidence contradicts it. We introduce TrustFork, an LLM agent safety benchmark with 1,890 tasks and 27,826 valid trajectories across 16 agent systems. These systems run eight orchestrators under the OpenCode, OpenClaw, and Pi harnesses. In each task, one subagent carries a risky goal while the other three stay aligned with the user, so contradicting evidence can exist. A task can also change the identity a subagent displays without changing the model behind it, which lets us trace a shift in authority to the label. Our analysis shows that even when another subagent contradicts the risky response, the orchestrator still acts on it in 72.0% of cases on average, most often in the systems with the least terminal harm. Swapping the family labels nearly triples how often the orchestrator obtains the risky response. A safer response is available in 84.0% of tasks, yet it decides the outcome in only 25.0%. The harness also decides which responses reach the orchestrator. Among three runtime defenses, hiding identity cues helps most consistently, while verifying before action helps only when the harness returns enough evidence. TrustFork shows that production agent orchestration must bind authority to evidence before execution causes harm. Our project is in https://henrymao2004.github.io/agent-orchestration-safety/.

相关论文

链接