CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

TL;DR

CompactionRL is a reinforcement learning strategy that integrates context compaction to improve the performance of long-horizon agentic LLMs.

Problem

Long-horizon agentic tasks can exceed the maximum context length before completion due to extended interaction trajectories, limiting the effectiveness of standard RL approaches.

Approach

The method combines context compaction with RL, jointly optimizing task execution and summary generation using token-level loss normalization and cross-trajectory generalized advantage estimation (GAE).

Results & Contribution

CompactionRL achieved consistent performance gains on agentic coding tasks with open models (GLM-4.5-Air, GLM-4.7-Flash), showing significant Pass@1 improvements on SWE-bench Verified and Terminal-Bench 2.0. It is also deployed in the RL pipeline for training the open GLM-5.2 model.