TL;DR
This paper proposes a digital teleoperation system that generates large-scale, high-quality robot learning data using a generative world model, eliminating the need for physical robots during data collection.
Problem
Collecting large-scale, diverse trajectory data for robot learning is severely bottlenecked by physical teleoperation, which ties operator time to specific hardware and workspaces.
Approach
Digital Teleoperation Paradigm: Replaces the physical robot with a generative world model to decouple data collection from physical constraints.
RynnWorld-Teleop System: Integrates depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer, and streaming autoregressive distillation.
Real-time Synthesis Pipeline: Compresses the generative process into a single-pass inference, enabling real-time interactive generation at 40+ FPS on a single H100 GPU.
Embodiment-Agnostic Labeling: The recorded pose stream serves as an embodiment-agnostic action label transferable to any target robot via standard retargeting.
Results & Contribution
Real-time High-Fidelity Generation: Achieves real-time interactive video generation at 40+ FPS on a single H100 GPU.
Zero-Shot Sim2Real Transfer: Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks.