智能体与自主科学 突破级 暂无讲解视频
发表时间
2026-05-13
arXiv
2605.07177

收录解读

这篇论文切中 multimodal search agent 的瓶颈:搜索质量和计算效率必须同时优化,尤其在并行多模态检索/观察场景中。

HyperEyes 的贡献在于用 dual-grained efficiency-aware RL 训练 parallel multimodal search agents。它把搜索 agent 的效率、粒度和决策策略放进统一训练目标。

它值得正式收录,因为 agentic search 正从文本检索转向多模态、并行、长期任务,本库已经收录 direct corpus interaction 等搜索范式论文,HyperEyes 补的是多模态并行搜索控制。

它没有更高,是因为需要看其方法是否能迁移到真实 web / enterprise / scientific multimodal search,而不是只在固定 benchmark 上有效。

原始摘要与中文对照

中文对照翻译

现有的多模态搜索代理按顺序处理目标实体,每个实体发出一次工具调用,并且当查询自然分解为独立的子检索时,会累积冗余的交互轮次。对于此类可分解查询,我们认为有效的多模态代理应该搜索更广而不是更长:在同一轮次内并发调度多个基于视觉的查询,而不是按顺序进行。为此,我们提出了HyperEyes,一个并行多模态搜索代理,它将视觉基础和检索融合到一个单一的原子动作中,从而实现跨多个实体的并发搜索,同时将推理效率视为首要的训练目标。HyperEyes分两个阶段进行训练:对于冷启动监督,我们开发了一个并行适用数据合成管道,涵盖视觉多实体和文本多约束查询,并通过渐进式拒绝采样策划了面向效率的轨迹。在此基础上,我们的核心贡献——一个双粒度效率感知强化学习框架——在两个互补的层面运作。在宏观层面,我们提出了TRACE(工具使用参考自适应成本效率),这是一种轨迹级奖励,其参考在训练期间单调收紧,以抑制多余的工具调用,同时不过度限制真正的多跳搜索。在微观层面,我们将策略蒸馏(OPD)应用于多模态代理搜索设置,在失败的推演中从外部教师注入密集的令牌级纠正信号,以减轻稀疏结果奖励的信用分配不足。由于大多数现有的多模态搜索基准仅将准确性作为唯一指标,忽略了推理成本和并行搜索能力,我们进一步引入了IMEB,这是一个人工策划的基准,它联合评估多模态搜索能力和效率,包含300个多实体视觉实例。在六个基准测试中,HyperEyes-30B在准确性方面超越了同等规模最强的开源多模态搜索代理9.9%,平均工具调用轮次减少了5.3倍。代码和数据可在https://github.com/DeepExperience/HyperEyes公开获取。

原始摘要

Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds whenever a query naturally decomposes into independent sub-retrievals. For such decomposable queries, we argue that effective multimodal agents should search wider rather than longer: dispatching multiple grounded queries concurrently within a round, rather than sequentially. To this end, we present HyperEyes, a parallel multimodal search agent that fuses visual grounding and retrieval into a single atomic action, enabling concurrent search across multiple entities while treating inference efficiency as a first-class training objective. HyperEyes is trained in two stages: for cold-start supervision, we develop a Parallel-Amenable Data Synthesis Pipeline covering visual multi-entity and textual multi constraint queries, and curate efficiency-oriented trajectories via Progressive Rejection Sampling. Building on this foundation, our central contribution, a Dual-Grained Efficiency-Aware Reinforcement Learning framework, operates at two complementary levels. At the macro level, we propose TRACE (Tool-use Reference-Adaptive Cost Efficiency), a trajectory-level reward whose reference is monotonically tightened during training to suppress superfluous tool calls without over-restricting genuine multi-hop search. At the micro level, we adapt On-Policy Distillation (OPD) to the multimodal agentic search setting, injecting dense token-level corrective signals from an external teacher on failed rollouts to mitigate the credit-assignment deficiency of sparse outcome rewards. Since most existing multimodal search benchmarks evaluate accuracy as the sole metric, omitting inference cost and parallel-search capability, we further introduce IMEB, a human-curated benchmark that jointly evaluates multimodal search capability and efficiency, comprising 300 multi-entity visual instances. Across six benchmarks, HyperEyes-30B surpasses the strongest open-source multimodal search agent of comparable scale by 9.9% in accuracy with 5.3× fewer tool-call rounds on average. Code & Data are publicly available at https://github.com/DeepExperience/HyperEyes.

链接