安全、治理与可靠性 颠覆级 有讲解视频
发表时间
2026-08-01
arXiv
2608.00677

收录解读

持久化环境中的早期状态变化会影响长程智能体行为,短静态基准无法捕捉这种累积风险。

OpenART通过环境演化进行开放式红队测试,并提出反馈驱动、无需参数更新的黑盒EMHA攻击策略。

平台含超过一万个状态场景、50个领域、50万工具技能和75种智能体模型配置,任务中位数为97次工具调用。

EMHA聚合攻击成功率85%,相对指令演化的优势随复杂度由约2%升至超过17%,显示运行时实现是独立安全变量。

原始摘要

AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-model interactions, agent behavior is mediated through shared state that is repeatedly read, modified, and reused across long-horizon workflows. Safety therefore depends not only on individual actions, but also on how agents respond as environments evolve over time. Existing agent safety benchmarks primarily evaluate short, static tasks, making it difficult to study cumulative risks in evolving environments; moreover, benchmark-specific interfaces hinder direct comparison across agent runtimes. To address these limitations, we introduce OpenART, an open-ended arena for scalable agent red teaming through environment evolution. OpenART constructs over 10K validated stateful scenarios spanning 50 domains from more than 500K Tools, MCPs, and Skills. The resulting tasks require a median of 97 tool calls and are projected through target adapters to 15 deployed agents, 5 foundation models, and 8 attack vectors, enabling unified evaluation across 75 agent–model configurations. To systematically explore evolving attack surfaces, we further propose the Evolutionary Markov Hypergraph Attack (EMHA), a black-box policy that performs feedback-driven environment evolution by coordinating authorized state transitions over hypergraph paths without parameter updates. Throughout evolution, task objectives and safety contracts remain fixed while only the environment state changes. Across all 75 agent–model configurations, EMHA achieves a pooled strict Attack Success Rate (ASR) of 85.0%. Its advantage over instruction-only evolution increases from 1.8–2.7% on simple environments to 17.2–17.6% on the most complex ones, demonstrating that environment evolution increasingly exposes safety failures as task complexity grows. Furthermore, incorporating target-agent identity explains an additional 7.6% of ASR variation beyond model and capability controls, suggesting that runtime implementation plays a significant role in agent safety. These results establish OpenART as a scalable foundation for studying agent safety in complex, evolving environments.

解读视频

相关论文

链接