Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games

TL;DR

We introduce RNG-Bench, a benchmark suite for evaluating multimodal LLMs' ability to reconstruct past observations and act on them in non-Markov games.

Problem

Existing benchmarks either expose the full state, conflate hidden-state reconstruction with other skills, or test recall only after an episode ends. Deploying multimodal foundation models as closed-loop policies requires conditioning actions on observations that are no longer visible.

Approach

RNG-Bench includes two complementary games (Matching Pairs and 3D Maze) with three controlled difficulty axes: grid size, visual pattern, and observation modality. It introduces a head-to-head duel protocol to control instance-level variance and a Memory Gap metric that disentangles forgetting from poor action selection.

Results & Contribution

The hardest configurations require ~128K tokens and 350 image inputs per episode and remain far from saturated by frontier MLLMs. Memory Gap analysis shows most residual errors stem from forgetting earlier observations rather than suboptimal decision making. Fine-tuning Qwen3.5-9B on optimal-policy rollouts improves RNG-Bench performance and transfers to existing benchmarks without degrading general multimodal capability.