Rethinking Generalization in Reasoning SFT (April 2026)

Free

Challenges "SFT memorizes, RL generalizes" — reasoning SFT with long CoT does generalize cross-domain, conditional on optimization dynamics; discovers safety-reasoning tradeoff (reasoning improves but safety degrades); 152 HF likes

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About Rethinking Generalization in Reasoning SFT (April 2026)

This preprint challenges the prevailing narrative that supervised finetuning (SFT) memorizes while reinforcement learning (RL) generalizes in LLM post-training. Through a conditional analysis of reasoning SFT with long chain-of-thought (CoT) supervision, the authors demonstrate that cross-domain generalization is not absent but conditional—jointly shaped by optimization dynamics, training data quality and structure, and base-model capability. Key findings include a dip-and-recovery pattern (performance first degrades then improves with extended training), the importance of verified long-CoT traces for consistent gains, and an asymmetric generalization where reasoning improves but safety degrades. The work reframes the question from whether reasoning SFT generalizes to under what conditions and at what cost.

Key Features

Cross-domain generalization in reasoning SFT is conditional, not absent

Dip-and-recovery pattern: performance degrades before improving with longer training

Data quality and structure (e.g., verified long-CoT traces) are critical for generalization

Stronger base models internalize transferable procedural patterns (e.g., backtracking)

Asymmetric generalization: reasoning improves but safety degrades

Identifies under-optimization artifacts as cause of reported generalization failures

Pros & Cons

Pros

Provides nuanced empirical analysis challenging oversimplified narratives
Identifies practical conditions (optimization, data, model) that enable generalization
Reveals dip-and-recovery pattern, warning against premature evaluation
Highlights asymmetric generalization, raising important safety considerations
Based on rigorous experimental methodology with long-CoT supervision

Cons

Preprint under review, not yet peer-reviewed
Safety degradation is a nontrivial cost of reasoning improvement
Generalization findings may depend on specific task domains and model scales
Requires access to verified long-CoT data, which may not be widely available

Best For

Informing LLM post-training strategies with reasoning SFTUnderstanding conditions for cross-domain generalization in AI modelsGuiding data curation for reasoning tasks (e.g., verified CoT traces)Analyzing safety-reasoning tradeoffs in model alignmentBenchmarking optimization dynamics in supervised finetuning

FAQ

What does this paper challenge?

It challenges the narrative that supervised finetuning (SFT) only memorizes while reinforcement learning (RL) generalizes, showing that reasoning SFT with long chain-of-thought supervision can generalize cross-domain under specific conditions.

What is the dip-and-recovery pattern?

Cross-domain performance first degrades early in training before recovering and improving with extended optimization, meaning short training checkpoints can underestimate generalization.

What conditions enable generalization in reasoning SFT?

Generalization is conditional on optimization dynamics (enough training steps), data quality and structure (verified long-CoT traces help), and base-model capability (stronger models internalize transferable patterns).

What is the safety-reasoning tradeoff?

The paper finds that while reasoning performance improves, safety alignment degrades—an asymmetric generalization that reframes the cost of reasoning SFT.