LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
FreeStructure, not content, drives successful reasoning in LLMs.
About LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
This paper from February 2025, authored by Dacheng Li and 11 others, demonstrates that large language models (LLMs) can effectively learn long chain-of-thought (Long CoT) reasoning through data-efficient supervised fine-tuning (SFT) and parameter-efficient LoRA. Using only 17k Long CoT training samples, the Qwen2.5-32B-Instruct model achieves dramatic improvements on math and coding benchmarks (e.g., 56.7% on AIME 2024, up 40%; 57.0% on LiveCodeBench, up 8.1%), competitive with the proprietary o1-preview model. The key insight is that the structure of reasoning demonstrations matters far more than the content: perturbations that maintain logical consistency (e.g., training on incorrect answers) have minimal impact, while structural disruptions (e.g., shuffling steps) severely degrade performance. The paper's released model and code (Sky-T1-32B-Preview) are open-sourced.
Key Features
Pros & Cons
- Achieves strong reasoning performance with very few (17k) training examples
- Uses efficient LoRA fine-tuning, reducing computational cost
- Outperforms many larger models and rivals proprietary o1-preview on math and coding tasks
- Reveals important insight that reasoning structure matters more than content, guiding future training
- Fully open-source model and code for reproducibility
- Limited evaluation to Qwen2.5-32B base model; generalizability to other architectures unknown
- Requires generation of long chain-of-thought demonstrations, which may be complex to create
- Performance on non-math/coding reasoning tasks not extensively tested