收录解读
ECHO 把 terminal agents 的环境交互视为可学习世界模型问题：命令执行后的文件系统、stdout、stderr 和状态变化可以作为自监督信号。
论文通过辅助预测环境后继状态，让 agent 在完成任务的同时学习 CLI world model，不需要额外人工标注即可改善后续规划。
这对 coding agents 和本地自动化尤其重要，因为终端环境有明确动作、观测和可验证状态，是 agent 世界模型最现实的落地点之一。
它值得正式收录，因为它把 world modeling 从机器人/视频扩展到软件执行环境，为 terminal agent 的长期学习提供了低成本训练信号。
原始摘要与中文对照
中文对照翻译
ECHO：终端智能体免费学习世界模型。CLI智能体是语言模型最接近具身环境的存在：模型发出命令，终端执行它们，并且返回的流——标准输出、错误、文件、日志和追踪——记录了结果。我们认为这个流是一个监督信号，但标准的智能体强化学习却将其丢弃：GRPO风格的训练使用稀疏的结果级奖励更新动作令牌，同时忽略了已在rollout中存在的环境响应。失败的rollout提供了很少的策略梯度信号，尽管它们包含了关于环境如何响应的丰富证据。我们引入了ECHO（环境交叉熵混合目标），这是一种混合目标，它将动作令牌上的标准策略梯度损失与一个辅助损失相结合，该辅助损失训练策略预测其自身动作产生的环境观察令牌。ECHO重用了与GRPO相同的正向传播，不需要额外的rollout，并将终端反馈转化为所有rollout的密集监督。ECHO使GRPO在TerminalBench2.0上的pass@1翻倍：Qwen3-8B从2.70%提升到5.17%，Qwen3-14B从5.17%提升到10.79%。ECHO还生成了能更好预测终端动态的策略，即使是在它们未生成的轨迹上：在保留的rollout中，它显著降低了环境令牌交叉熵，而单独的GRPO几乎没有改变。从基础Qwen3-8B开始，ECHO在没有专家演示的情况下，在保留的终端任务上达到了专家SFT-然后-GRPO的性能，并恢复了TerminalBench-2.0上专家SFT初始化收益的大约一半。在某些设置中，仅环境预测损失就能实现无验证器的自我改进，允许策略仅通过学习环境交互来改进未见的OOD任务。综上所述，这些结果表明环境观察不仅仅是未来行动的上下文，而是每个rollout中已经存在的密集、在策略监督信号。
原始摘要
CLI agents are the closest thing language models have to an embodied setting: the model emits commands, the terminal executes them, and the returned stream—stdout, errors, files, logs, and traces—records the consequences. We argue that this stream is a supervision signal, but standard agent RL discards it: GRPO-style training updates action tokens with sparse outcome-level rewards while ignoring environment responses already in the rollout. Failed rollouts provide little policy-gradient signal despite containing rich evidence about how the environment responds. We introduce ECHO (Environment Cross-entropy Hybrid Objective), a hybrid objective that combines the standard policy-gradient loss on action tokens with an auxiliary loss that trains the policy to predict environment observation tokens resulting from its own actions. ECHO reuses the same forward pass as GRPO, requires no additional rollouts, and turns terminal feedback into dense supervision for all rollouts. ECHO doubles GRPO pass@1 on TerminalBench2.0: Qwen3-8B improves from 2.70% to 5.17%, and Qwen3-14B from 5.17% to 10.79%. ECHO also produces policies that better predict terminal dynamics, even on trajectories they did not generate: across held-out rollouts, it sharply reduces environment-token cross-entropy while GRPO alone barely changes it. From base Qwen3-8B, ECHO matches expert-SFT-then-GRPO performance on held-out terminal tasks without expert demonstrations, and recovers roughly half of the expert-SFT initialization benefit on TerminalBench-2.0. In some settings, the environment prediction loss alone enables verifier-free self-improvement, allowing policies to improve on unseen OOD tasks by learning only from environment interactions. Together, these results suggest that environment observations are not merely context for future actions, but a dense, on-policy supervision signal already present in every rollout.