Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning

TL;DR

RA-RFT is a post-training framework that combines retrieval augmentation and reinforcement fine-tuning to enable language models to perform analogical reasoning.

Problem

Conventional RAG relies on semantic similarity for retrieval, which is ill-suited for complex reasoning tasks: semantically similar problems may require different solution strategies, while superficially different problems may share the same underlying reasoning pattern. Thus, a retrieval method specialized for reasoning is needed.

Approach

RA-RFT consists of two stages. First, it trains a retriever via gold-relevance distillation to rank contexts by expected reasoning benefit rather than semantic overlap. Second, it fine-tunes the policy model via reinforcement fine-tuning methods with retrieved analogous demonstrations, learning to leverage reasoning traces under verifiable outcome rewards.

Results & Contribution

On mathematical reasoning benchmarks (e.g., AIME 2025), RA-RFT consistently outperforms standard reinforcement fine-tuning methods like GRPO. For instance, it improves AIME 2025 average@32 accuracy by 7.1 and 2.8 points over GRPO for Qwen3-1.7B and Qwen3-4B respectively. This demonstrates that reasoning-aware retrieval is an orthogonal axis of improvement to reward design or training curricula.