Reasoning Quality Emerges Early

Analysis

Why This Paper Matters

This paper addresses a critical bottleneck in eliciting strong reasoning capabilities from Large Language Models (LLMs): the costly and often suboptimal curation of high-quality supervised fine-tuning (SFT) data. Current methods depend on strong reasoning models to filter examples based on diversity and difficulty, which is expensive and may yield limited data quality. The authors propose a novel, efficient approach that leverages only the initial reasoning tokens to identify difficult and diverse examples, eliminating the need for strong reasoning models. This is significant because it dramatically reduces the computational overhead of data curation—achieving 91% token efficiency—while still improving performance by up to 1.7% over existing baselines. For AI practitioners, this means more accessible and scalable fine-tuning pipelines for reasoning tasks, particularly in domains like medical reasoning where high-quality data is scarce.

Technical Contributions

The paper makes several key technical innovations:

• Early token loss as difficulty proxy: Demonstrates that the loss of the first 100 reasoning tokens at a randomly perturbed checkpoint of the pretrained model reliably detects difficult problems. This is a simple yet effective heuristic that avoids expensive full-sequence evaluation.
• Gradient similarity via loss patterns: Proves that examples with similar loss patterns over their first 1k reasoning tokens across a small number of perturbed checkpoints (extrapolating along the fine-tuning trajectory) induce similar gradients. This provides a principled way to group examples for efficient SFT.
• Token-efficient curation: The method requires only a fraction of the tokens (first 100-1k) compared to full-sequence analysis, leading to 91% token efficiency without sacrificing quality.
• Empirical validation on diverse models and datasets: Tested on Qwen2.5-7B and Llama3.1-8B models using M23K medical reasoning and OpenThoughts-Math datasets, demonstrating robustness across domains.

Results

The proposed method outperforms existing baselines by up to 1.7% on the evaluated benchmarks. Specifically, on the M23K medical reasoning dataset, the approach achieves higher accuracy while using 91% fewer tokens for data curation. On OpenThoughts-Math, similar gains are observed. The token efficiency is a standout metric: reducing the computational cost of data curation by nearly an order of magnitude while still improving performance. This is a rare combination of efficiency and effectiveness.

Significance

This work has broad implications for the AI field. By reducing the dependency on strong reasoning models for data curation, it lowers the barrier to entry for fine-tuning LLMs on reasoning tasks. The approach is particularly impactful for resource-constrained settings (e.g., academic labs, small companies) where access to powerful models is limited. Furthermore, the theoretical insight that initial tokens carry sufficient signal for difficulty and diversity assessment may inspire new methods for efficient data selection in other domains beyond reasoning. The 91% token efficiency also has environmental benefits by reducing compute requirements. Overall, this paper advances the practical deployment of LLMs with strong reasoning capabilities.