MemoryWAM: Efficient World Action Modeling with Persistent Memory
Sizhe Yang Juncheng Mu Tianming Wei Chenhao Lu Xiaofan Li Linning Xu Zhengrong Xue Zhecheng Yuan Dahua Lin Jiangmiao Pang Huazhe Xu
作者信息
Abstract
Robust robotic manipulation in the real world requires not only an understanding of the current observation, but also memory and dynamics modeling. World action models (WAMs) possess these capabilities by jointly modeling visual foresight and actions conditioned on both current and historical observations, making them a promising paradigm for robotic manipulation. However, existing WAMs face a fundamental trade-off: methods with efficient inference typically condition only on a bounded window of recent observations and therefore struggle in non-Markovian environments, whereas methods that preserve long histories incur time and space costs that grow substantially with sequence length. To address this challenge, we introduce MemoryWAM, a world action model with efficient persistent memory. MemoryWAM uses a hybrid memory design that combines recent frames, event-boundary anchor frames, and compact gist tokens that summarize long-range history. A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context, supporting memory-dependent decision-making with reduced inference latency and GPU memory usage. Across long-horizon, memory-dependent manipulation tasks in both simulation and the real world, MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines while maintaining favorable computational efficiency.引用本文复制引用
Sizhe Yang,Juncheng Mu,Tianming Wei,Chenhao Lu,Xiaofan Li,Linning Xu,Zhengrong Xue,Zhecheng Yuan,Dahua Lin,Jiangmiao Pang,Huazhe Xu.MemoryWAM: Efficient World Action Modeling with Persistent Memory[EB/OL].(2026-06-18)[2026-06-21].https://arxiv.org/abs/2606.20562.学科分类
计算技术、计算机技术/自动化技术、自动化技术设备
|国家预印本平台