收录解读
多智能体博弈算法(如CFR和PSRO)长期依赖人工设计与启发式调优,新算法的发现需要大量领域专业知识。本文将算法设计重新定性为符号代码空间上的搜索问题,希望借助LLM的推理能力自动化这一发现过程,从而摆脱对人工直觉的依赖。
论文采用AlphaEvolve框架,将算法源代码视为可演化的"基因组",由LLM驱动的进化智能体在组合符号操作空间中搜索,产生了VAD-CFR(波动自适应折扣CFR)和SHOR-PSRO两个新算法。更关键的是提出了"算法蒸馏"方法:从LLM生成的复杂、环境特定解中提炼出最小核心机制,大幅降低复杂度并提升泛化性。
本文展示了LLM不仅能写代码,还能自主"发现"此前人类未曾设计出的博弈论算法,且在18个不同游戏上验证了泛化能力,标志着自主科学发现范式在算法设计领域的实质性推进。对于本仓库而言,这是"LLM作为科学家"路线在多智能体博弈方向的高质量落地案例,具有重要的方法论示范价值。
当前评估局限于精确best-response oracle,尚未测试基于RL的实际部署场景;演化出的机制有时对人类而言"非直觉",可解释性有限。泛化测试也仅限于游戏变体,未延伸至本质不同的领域,限制了其通用性声称的边界。
原始摘要
Much of the advancement in Multi-Agent Reinforcement Learning (MARL) for imperfect-information games has historically depended on the manual, iterative refinement of algorithmic baselines. Recently, evolutionary coding agents powered by Large Language Models (LLMs) have emerged as powerful tools to automate this discovery process. In this work, we deploy one of such agentic frameworks, AlphaEvolve, to navigate the design spaces of two distinct game-theoretic paradigms: counterfactual regret minimization (CFR) and policy-space response oracles (PSRO). This automated search yielded two algorithms: Volatility-Adaptive Discounted (VAD-) CFR and Smoothed Hybrid Optimistic Regret (SHOR-) PSRO, which are consistently competitive with state-of-the-art human-designed baselines across an 18-game evaluation suite spanning Poker, Goofspiel, Liar's Dice, Blotto, and Battleship variants. However, because the LLM optimizes for fitness on a specific training set, it often constructs highly synergistic, complex mechanisms tailored to those environments. Through systematic ablation studies, we demonstrate that while these mechanisms are tightly coupled, the true driver of generalization lies in a minimal algorithmic core. By distilling the LLM's discoveries down to their most fundamental principles, we produce two minimal solvers: Warm-started Optimistic Predictive (WOP-)CFR and Projection Matching (PM-)PSRO. These distilled versions achieve superior performance on generalization with greatly reduced structural complexity, providing a clear methodology for using LLMs in algorithmic discovery.