收录解读
当前学界普遍认为，基于可验证奖励的强化学习（RLVR）能够使大语言模型超越基础模型的能力边界，在数学、编程等推理任务上持续自我提升、涌现出全新的推理能力。然而，这一假设从未在系统层面得到严格验证。本文以"RLVR是否真正激发了超越基础模型的推理容量"为核心问题，对多个模型家族、多种RL算法及多类推理基准展开全面实证分析。
论文引入pass@k指标作为核心评估工具，在大k值下（k=128至1024）系统比较RLVR训练模型与基础模型的覆盖能力。关键发现是：RLVR训练模型仅在小k（如k=1）时超过基础模型，当k增大时基础模型始终反超，说明RLVR提升的是"采样效率"而非"推理边界"。此外，困惑度分析证实RLVR模型的推理路径已经存在于基础模型的输出分布之中，而对比实验表明蒸馏方法才能真正引入基础模型中不存在的新推理模式。
这项工作对当前AI推理研究的主流叙事构成实质性挑战：大量将RLVR视为"激发新推理能力"的论文需要重新审视其结论的有效范围。该论文提供了清晰的方法论框架（pass@k + 困惑度分析），可作为评估未来推理增强方法的标准工具，对本仓库关注的推理与推断控制方向具有重要参考价值。
该论文的主要局限在于其结论高度依赖当前RLVR的具体实现（PPO、GRPO等六种算法），对更前沿或未来改进的RL范式尚未覆盖。作者本人也指出研究停留在诊断层面，未提出足以弥补"推理边界"差距的具体新方法。此外，实验规模（最大k=1024）受计算资源限制，极端k值下的结论是否成立仍有讨论空间，这是其未达到更高评级的原因。
原始摘要
Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs), particularly on mathematics and programming tasks. Similar to how traditional RL helps agents explore and learn new strategies, RLVR is believed to enable LLMs to continuously self-improve, thus acquiring novel reasoning abilities beyond those of the corresponding base models. In this study we critically examine the current state of RLVR by systematically probing the reasoning capability boundaries of RLVR-trained LLMs across various model families, RL algorithms, and math, coding, and visual reasoning benchmarks, using pass@k at large k values as the evaluation metric. Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model. Treating the base model as an upper bound, our quantitative analysis shows that six popular RLVR algorithms perform similarly and remain far from optimal in leveraging the potential of the base model. By contrast, we find that distillation can introduce new reasoning patterns from the teacher and genuinely expand the model's reasoning capabilities. Overall, our findings suggest that current RLVR methods have not yet realized the potential of RL to elicit truly novel reasoning abilities in LLMs. This highlights the need for improved RL paradigms, such as continual scaling and multi-turn agent-environment interaction, to unlock this potential.