Jie Zhang, Xiaoyue Chen, Anzhe Chen, Chenxu Lv, Deqing Li, Gengze Zhou, Hang Yin, Haoqi Yuan et al.
Qwen-RobotWorld is a unified video world model that generates future visual trajectories for diverse embodied tasks (robotic manipulation, autonomous driving, indoor navigation, human-to-robot transfer) using natural language as action commands.
Existing embodied world models are limited to specific tasks or domains, lacking generalization and flexible control via natural language. Additionally, unifying multiple applications such as synthetic data generation for policy learning, evaluation environment construction, and planning signal provision into a single model remains a challenge.
1) Double-Stream MMDiT with MLLM Action Encoding: A 60-layer double-stream diffusion transformer couples frozen Qwen2.5-VL semantics with video-VAE latents through layer-wise joint attention. 2) Embodied World Knowledge (EWK): An 8.6M video-text corpus (200M+ frames) covering 20+ embodiments and 500+ action categories is built to learn action-language mapping. 3) General+Expert Progressive Curriculum: A two-stage training strategy that first learns general visual priors and then injects embodied specialization under a shared language interface.
Achieves 1st overall on EWMBench and DreamGen Bench, outperforms all open-source models on WorldModelBench and PBench. Zero-shot analyses on RoboTwin-IF benchmark demonstrate robust generalization and multi-view consistency. Proposes three application directions: synthetic data generation, virtual environment evaluation, and language-guided planning signals, significantly enhancing the scalability of embodied AI.