智能体与自主科学 颠覆级 有讲解视频
发表时间
2026-07-21
arXiv
2607.16387

收录解读

智能体系统执行轨迹记录了失败模式,但原始轨迹冗长、实例化且缺乏稳定词汇,不利于积累反馈。现有方法利用轨迹进行搜索、提示优化和运行时监控,但缺乏统一的故障表示。

本文提出AdaMAST,从智能体轨迹中自动诱导紧凑、基于证据的失败分类法:沿系统级、角色特定和领域特定三个轴组织故障代码,所有名称和定义均由数据自动生成,无需人工注释。该分类法作为共享反馈接口,可用于智能体搜索、运行时修复和轨迹选择。

实验表明:在SWE-bench Verified Mini上,分类法反馈将SWE-agent的解决率从60%提升至70%,Claude Code从64.0%提升至70.7%;在Terminal-Bench 2.0上,基于分类法的验证器将best-of-5准确率提高8-15个点。分类法在保持高区分度的同时实现数量级压缩,且与专家标注的一致性优于手工词汇。

自适应失败分类法弥合了轨迹生成与改进过程之间的鸿沟,为智能体调试提供了一种可重用、可扩展的范式。但当前验证集中在代码和终端任务,未来需扩展到更广泛的智能体系统和物理世界场景。

原始摘要

An agent system's execution traces record how it fails, and procedures that improve such a system without changing model weights (trajectory selection, prompt and workflow optimization, runtime monitoring) read these traces for feedback. Yet raw traces are a poor medium for accumulating that feedback: long, instance-specific, and lacking a stable vocabulary for recurring failures. We argue that an agent system should instead maintain an explicit representation of how it fails, induced from its own behavior and reusable wherever failure feedback is needed. AdaMAST builds this representation by converting a target system's traces into a compact, evidence-grounded failure taxonomy: named failure codes organized along three fixed axes (system-level, role-specific, and domain-specific), with every name, definition, and evidence pattern induced from the traces; no code is hand-authored, no trace human-annotated. The taxonomy is not merely a post-hoc diagnostic but a shared feedback interface, improving agents in three ways. In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form reflection on all five benchmarks we test. At runtime, taxonomy feedback raises SWE-agent's resolution on SWE-bench Verified Mini from 60% with free-text reflection to 70%, and improves Claude Code from 64.0% to 70.7% as a runtime skill. In trajectory selection, AdaMAST-Judge, a verifier built on the induced codes, improves best-of-5 accuracy on Terminal-Bench 2.0 by 8-15 points over Pass@1. The vocabulary itself is compact (an order-of-magnitude compression that preserves trace distinctions), human-faithful (matching expert failure annotations more closely than a hand-crafted reference vocabulary), and adaptive (taxonomies induced for different domains share few codes). Adaptive failure taxonomies close the loop between the traces agents produce and the procedures that improve them.

解读视频

相关论文

链接