Reasoning or retrieval? a study of answer attribution on large reasoning models

Analysis

Why This Paper Matters

Large reasoning models (LRMs) have become central to AI applications, yet their reliance on parametric knowledge versus external retrieval remains poorly understood. This paper addresses a critical gap by systematically studying answer attribution—whether an LRM's answer stems from its internal reasoning or from retrieved evidence. Understanding this is essential for building trustworthy systems, especially in knowledge-intensive tasks where factual accuracy is paramount.

The findings challenge the assumption that retrieval-augmented generation (RAG) always improves LRM performance. By showing that LRMs often favor reasoning over retrieval, the paper highlights a potential failure mode: models may ignore correct retrieved evidence in favor of plausible but incorrect reasoning. This has direct implications for RAG system design and for debugging model behavior.

Technical Contributions

The paper introduces a novel evaluation framework that disentangles reasoning and retrieval contributions. Key technical elements include:

• Controlled question-answering tasks with conflicting reasoning and retrieval signals.
• Attribution analysis using output probabilities and attention patterns.
• Probing experiments to measure internal representation of retrieved evidence.
• Systematic variation of reasoning confidence and retrieval relevance to map usage boundaries.

Results

While the abstract is truncated, the described experiments reveal that LRMs attribute answers to reasoning more frequently than to retrieval. Specifically, retrieval is more likely to be used when reasoning confidence is low or when retrieved evidence is highly relevant. The paper does not report exact metrics in the abstract, but the qualitative findings are consistent across multiple task types. These results suggest a bias toward parametric knowledge, which could be problematic in dynamic or niche domains.

Significance

This research has broad implications for AI system design. It suggests that simply adding retrieval to LRMs is insufficient; models must be trained or prompted to appropriately weigh external evidence. The findings also inform interpretability efforts, as attribution analysis can help identify when models are over-relying on reasoning. Future work could extend this to multi-hop reasoning and real-world retrieval pipelines, and develop methods to calibrate the reasoning-retrieval balance.