SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation

TL;DR

SSD leverages 2D spatial correlation in images for speculative decoding, accelerating autoregressive image generation by up to 13.3x.

Problem

Autoregressive image generation models flatten images into 1D token sequences, discarding intrinsic 2D spatial locality. This creates memory bandwidth bottlenecks during inference, slowing generation.

Approach

Instead of predicting only the next token in 1D, SSD trains the model to simultaneously predict the right and bottom neighbor tokens. This exploits 2D spatial correlation to generate multiple tokens at once within a speculative decoding framework.

Results & Contribution

Achieves up to 13.3x speedup on DPG-Bench and GenEval while maintaining high fidelity. Demonstrates that respecting image geometry unlocks massive computational efficiencies, paving the way for real-time high-resolution autoregressive models.