收录解读
机器人长时程任务需要保留历史,而许多世界–动作模型只利用短观测窗口,部署时还面临视频预测的时延。
Long-WAM以自回归世界视频预训练建立对长历史的利用能力,再结合世界–动作训练及流式/异步运行;对历史长度、预训练方式与部署进行比较。
以自回归视频预训练让世界–动作模型利用长历史,并配套流式/异步部署;4仿真基准、G1/YAM两实机验证,RoboCasa0→19.2秒历史63.3%→78.7%,动态堆杯95%对若干基线0/20,RTX5090动作块107.4毫秒。
未达到paradigm:预训练约30720 H100 GPU小时且实机任务/试次有限,未证实廉价通用自主学习。
原始摘要
Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints. Our central finding is that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR). We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas a bidirectionally pretrained initialization shows no net gain; robot-domain AR pretraining further raises peak success on GR-1 and LIBERO-Long. Long-WAM also achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction; on RTX 5090, each action chunk, including future-video latent prediction, takes 107.4 ms. Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.