收录解读
AI模型能力强大但机制不清,安全性难以控制,现有机械可解释性探索主要靠人工,效率低。本文提出用AI自动化发现AI智能机制,以弥合理解与控制的鸿沟。
Mechanist系统构建了约1.3万篇可解释性论文的知识图谱,整合覆盖26个领域、4300万篇论文的多学科数据库,并整理32种基础分析方法。作为智能体,它能够自主生成假设、设计实验、验证机制,并与Claude Code等系统比较。
实验声称Mechanist相比Claude Code和现有AI科学家系统能生成更有价值的机制假说、更可靠地执行实验。它发现了跨模态不安全特征可通过看似安全的数据迁移的安全风险,提出了关于信念的机制理论,解释模型如何表征世界知识、形成信念、推测他人信念及其预训练涌现,并将机制见解转化为实际干预,提升多场景性能,引导科学基础模型生成指定DNA序列。
完整 PDF 给出 16 篇论文、9 个主题的盲复现实验,三位专家与两种模型评审均把 Mechanist 排在 Claude Code 和 AI Scientist 之前,并开放代码和数据;但仍缺少外部团队复现和跨科学领域长期检验。
原始摘要
AI models have achieved remarkable success across diverse domains, yet the mechanisms underlying their capabilities and the risks they may pose remain poorly understood. As AI development becomes faster and increasingly automated, mechanistic exploration remains largely manual, widening the gap between what models can do and our ability to understand and control them. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence. To support autonomous mechanistic discovery, we construct an interpretability-focused knowledge graph of approximately 13,000 papers and integrate it with a multidisciplinary database of 43 million papers spanning 26 fields. We further curate a library of 32 foundational methods for mechanism analysis, causal intervention, and validation. Compared with Claude Code and existing AI-scientist systems, Mechanist generates more valuable mechanism hypotheses and executes experiments more reliably. Mechanist also demonstrates a progression from discovering model behaviors to explaining and controlling AI models. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer across modalities through apparently safe training data. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Finally, Mechanist translates these mechanistic insights into practical interventions that improve model performance across diverse scenarios and steer scientific foundation models toward generating DNA sequences with specified properties.