Hongyuan Adam Lu, Z. L. Victor Wei, Qun Zhang, Jinrui Zeng, Bowen Cao, Lingwei Meng, Mocheng Li, Zezhong Wang et al.
LoopWM introduces a looped architecture for world models, iteratively refining latent states with a parameter-shared transformer block, achieving up to 100x parameter efficiency and adaptive computation depth.
Current world models face a trade-off: deep models are needed for long-horizon accuracy but are expensive and prone to error accumulation. Parameter efficiency and adaptive computation are required.
A single transformer block with shared parameters is applied iteratively to refine latent states. The number of iterations adapts to prediction complexity, providing a new scaling axis orthogonal to model size and data.
Achieves up to 100x parameter efficiency over conventional methods with adaptive computation. First to introduce looped architectures for world modeling, establishing iterative latent depth as a new scaling axis.