Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
FreeLearning How to Think With Meta Chain-of-Thought
About Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
This paper proposes Meta Chain-of-Thought (Meta-CoT), a novel framework that extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. The authors present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. They outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. The work discusses open research questions including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms, aiming to enable more powerful and human-like reasoning in LLMs.
Key Features
Pros & Cons
- Explicitly models underlying reasoning process beyond standard CoT
- Combines multiple advanced techniques (supervision, search, RL)
- Provides a concrete pipeline for implementation
- Backed by empirical observations from state-of-the-art models
- Proposed framework, not yet fully realized in practice (research stage)
- Training pipeline is complex and computationally expensive
- Open research questions remain, e.g., scaling laws and verifier roles
- No immediate implementation or code released in the paper