Recursive Language Models

Analysis

Why This Paper Matters

This paper addresses a critical bottleneck in large language models: the fixed context window. While recent models have expanded context lengths, they still fail on inputs that exceed their limits, and even within limits, performance degrades on long inputs. The authors propose Recursive Language Models (RLMs), a paradigm that shifts the focus from model architecture to inference-time computation. By allowing the model to recursively examine and process prompt snippets, RLMs effectively extend the usable input length by orders of magnitude, without requiring retraining or architectural changes.

The significance is twofold. First, it offers a practical solution for processing arbitrarily long documents, codebases, or multi-file projects, which are common in real-world applications. Second, it challenges the assumption that context length is a fixed property of the model, suggesting that with the right inference strategy, even small models can handle inputs far beyond their training context. This aligns with the growing interest in inference-time scaling, where compute is used at inference to improve quality and capability.

Technical Contributions

The core innovation is the recursive inference loop. Instead of feeding the entire prompt at once, the model is given a programmatic interface to inspect the prompt, select relevant snippets, and recursively call itself on those snippets. This allows the model to break down complex tasks into manageable subproblems, similar to how a human would approach a long document.

Key technical elements include:

• Environment abstraction: The prompt is treated as an external environment, not a fixed input tensor.
• Recursive self-calls: The model can invoke itself on sub-prompts, enabling hierarchical processing.
• Programmatic control: The model can use tools or code to examine and decompose the prompt, integrating with existing coding scaffolds.
• Post-training: The authors train RLM-Qwen3-8B, demonstrating that even a small model can be adapted to effectively use the recursive paradigm.

Results

The empirical results are striking. RLMs on GPT-5 achieve a median improvement of 26% over compaction, 130% over CodeAct with sub-calls, and 13% over Claude Code across four diverse long-context tasks, at comparable cost. This shows that RLMs are not just a niche trick but a robust improvement over strong baselines.

Even more impressive is the small-scale model: RLM-Qwen3-8B outperforms the base Qwen3-8B by 28.3% on average and approaches the quality of vanilla GPT-5 on three of the four tasks. This suggests that the recursive paradigm can be learned and that smaller models can benefit significantly, potentially democratizing access to long-context capabilities.

The ability to process inputs up to two orders of magnitude beyond the context window is a major leap, indicating that the approach scales well with input length.

Significance

This work opens a new direction for handling long contexts in LLMs. Rather than relying solely on architectural innovations like sparse attention or longer training, RLMs demonstrate that inference-time computation can be a powerful alternative. This could lead to more efficient use of existing models and reduce the need for ever-larger context windows.

The recursive paradigm also has implications for agentic AI, where models need to interact with external tools and data. By treating prompts as environments, RLMs blur the line between reasoning and acting, potentially enabling more sophisticated problem-solving.

Future work could explore combining RLMs with other inference-time techniques, such as chain-of-thought or self-consistency, and extending the post-training approach to larger models. The open-source code will facilitate further research and adoption.