安全、治理与可靠性 颠覆级 有讲解视频
发表时间
2026-08-31
arXiv
2608.27646

收录解读

该论文针对智能体继承人类凭证后可能越权访问、被隐藏指令操纵或将敏感数据带入模型上下文的问题,指出仅依赖提示词作为护栏并不可靠,因为同一个易错的推理者既要执行任务又要执行限制。

核心方法是在智能体推理之外设置受信任的工具边界,即Out-of-Band Policy Enforcement(OBPE):在向后端发起调用前授权类型化操作和资源、收窄查询,并在响应中过滤记录/字段或掩码值;语义门控还可基于参数值或外部状态拒绝或暂停调用。数据策略所有者设定最大授权,智能体策略只能收窄而不能放宽,并给出了策略计划顺序无关性及智能体策略不能扩大上限的形式化证明。

实验使用简化自生产系统的HTTP代理原型和Cedar策略核心,在Jira和ServiceNow模拟环境上比较四个模型、20个自适应红队任务,共3,621次试验:追踪失败率从57.6%降至0.2%,聚类加权降低41.2个百分点[95% CI: 27.7, 54.9];任务完成率从79.1%降至60.9%,但配对安全有用完成率上升21.8个百分点[9.5, 35.2]。

该工作为智能体工具边界的安全执行提供了可复用的方法、原型和对抗评测框架,具有较强的方法与工程价值;但其形式化保证依赖限定条件,且未能解决非干扰意义上的聚合推断或行数oracle问题,写入控制、持久审批、时间性和聚合策略也未纳入评测,因此尚未达到重塑整个领域的基础性范式。

原始摘要

Give an agent a human's credential and it inherits the person's reach without the judgment that limits its use. It can sweep every reachable record into model context, where hidden instructions steer its next call, and every request stays credential-valid while the agent exceeds its job or absorbs a secret. Prompts are a brittle guardrail: one fallible reasoner interprets the task and enforces its limits. We present Out-of-Band Policy Enforcement (OBPE), a trusted boundary outside agent reasoning. It authorizes the typed operation and resource, narrows the query before the backend call, then filters records and fields or masks values in the response. Semantic gating can deny or hold an authorized call on argument values or external state. A data policy owner sets the maximum grant; agent policy can only narrow it. We prove, under stated conditions, that the policy plan is order-independent and agent policy cannot widen the ceiling. Field removal covers one execution; masking and history rules claim less. We release an HTTP proxy prototype simplified from our production system, with conformance tests tying its typed Cedar policy core to the model. Against Jira and ServiceNow mocks, our benchmark compares prompted agents with and without OBPE on four models, including 20 adaptive red-team tasks. A trace failure means protected data entered agent context, an exact value appeared in the answer, or a forbidden effect completed. In 3,621 trials it fell from 57.6% to 0.2%, a cluster-weighted reduction of 41.2 points [95% CI: 27.7, 54.9]; fulfillment fell from 79.1% to 60.9%, while paired safe-useful completion rose 21.8 points [9.5, 35.2]. Some answers reconstructed a value that never entered context or used filtered row counts as an oracle: shaping one execution is not noninterference. Write controls, durable approval, and temporal and aggregate policies lie outside this evaluation.

解读视频

相关论文

链接