Self-Evolving World Models for LLM Agent Planning

TL;DR

This paper proposes a self-evolving world model framework that improves its context at test-time based on prediction errors to enhance LLM agent long-horizon planning.

Problem

World models help agents plan by predicting future outcomes, but unreliable predictions can degrade decision-making. Existing static world models fail to adapt to new environments or correct their own prediction errors.

Approach

WorldEvolver self-evolves the world model's context at deployment time via three modules: (i) Episodic Memory for retrieval-based simulation using real action transitions, (ii) Semantic Memory for extracting heuristic rules from prediction-observation mismatches, and (iii) Selective Foresight for filtering low-confidence predictions before agent reasoning. Crucially, it updates only the world model's context while keeping the agent and all model parameters frozen.

Results & Contribution

Experiments on ALFWorld and ScienceWorld show WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines in downstream agent success rates. This demonstrates that test-time memory revision can enhance both predictive fidelity and planning performance.