软件工程与编程智能体 颠覆级 有讲解视频
发表时间
2026-09-30
arXiv
2609.37686

收录解读

通用计算机操作的智能体已有进展,但专业工业工程软件自动化仍不可靠,因为工程工作流需要在CAD/CAE/CAM/BIM/EDA等不同软件和设计阶段之间保持几何与物理约束及依赖关系;现有评测缺乏覆盖完整设计回路的基准。

EngiWorld提出首个围绕完整设计回路的基准:包含1,301个专家任务,覆盖6个工程域、26个专业软件平台,并提供GUI与CLI接口和6种任务类型;同时引入工件中心评测方法,通过统一域验证器套件程序化检查中间/最终工件的几何有效性、物理可行性和规则符合性,并按规格达成度对定量设计任务连续评分。

对7个前沿模型的评测显示明显能力缺口:最强模型EngiScore仅44.3,多软件组合任务的端到端成功率仅3.6%,说明现有agent在工程软件上的自动化能力尚不能可靠完成跨阶段工作流。

该工作为专业工程软件端到端智能体提供了首个可量化评测底座,具有较高的基准与验证方法论价值;限制是公开材料仅能确认基准设计,未见代码/数据/验证器发布细节,也尚未展示后续社区使用或模型能力提升的证据。

原始摘要

Autonomous agents have made rapid progress in general-purpose computer use, but reliable automation of professional industrial engineering remains out of reach, as engineering workflows demand reasoning over geometric and physical constraints and dependencies preserved across software and design stages. We present EngiWorld, the first benchmark structured around the complete design loop: 1,301 expert-curated tasks spanning 6 engineering domains (CAD, CAE, CAM, BIM, EDA, and 3D visualization) and 26 professional software platforms, with both GUI and CLI interfaces and 6 task types ranging from software-selection to open-ended tasks. We further introduce an artifact-centric evaluation methodology built on a unified domain-verifier suite, which programmatically checks the geometric validity, physical feasibility, and rule compliance of final and intermediate artifacts, and scores quantitative design tasks continuously by specification attainment rather than binary success. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model achieves an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeed. EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.

解读视频

相关论文

链接