|国家预印本平台
首页|Martian World Models: Controllable Video Synthesis with Physically Accurate 3D Reconstructions

Martian World Models: Controllable Video Synthesis with Physically Accurate 3D Reconstructions

Martian World Models: Controllable Video Synthesis with Physically Accurate 3D Reconstructions

来源:Arxiv_logoArxiv
英文摘要

Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-quality Martian data and the significant domain gap between Martian and terrestrial imagery. To address these challenges, we propose a holistic solution composed of two key components: 1) A data curation pipeline Multimodal Mars Synthesis (M3arsSynth), which reconstructs 3D Martian environments from real stereo navigation images, sourced from NASA's Planetary Data System (PDS), and renders high-fidelity multiview 3D video sequences. 2) A Martian terrain video generator, MarsGen, which synthesizes novel videos visually realistic and geometrically consistent with the 3D structure encoded in the data. Our M3arsSynth engine spans a wide range of Martian terrains and acquisition dates, enabling the generation of physically accurate 3D surface models at metric-scale resolution. MarsGen, fine-tuned on M3arsSynth data, synthesizes videos conditioned on an initial image frame and, optionally, camera trajectories or textual prompts, allowing for video generation in novel environments. Experimental results show that our approach outperforms video synthesis models trained on terrestrial datasets, achieving superior visual fidelity and 3D structural consistency.

Longfei Li、Zhiwen Fan、Wenyan Cong、Xinhang Liu、Yuyang Yin、Matt Foutter、Panwang Pan、Chenyu You、Yue Wang、Zhangyang Wang、Yao Zhao、Marco Pavone、Yunchao Wei

航空航天技术航天测绘学计算技术、计算机技术

Longfei Li,Zhiwen Fan,Wenyan Cong,Xinhang Liu,Yuyang Yin,Matt Foutter,Panwang Pan,Chenyu You,Yue Wang,Zhangyang Wang,Yao Zhao,Marco Pavone,Yunchao Wei.Martian World Models: Controllable Video Synthesis with Physically Accurate 3D Reconstructions[EB/OL].(2025-07-10)[2025-07-23].https://arxiv.org/abs/2507.07978.点此复制

评论