Neura Market
Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers logo

Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

Free

Self-play mutual reasoning to boost small LLMs without fine-tuning

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Visit Website GitHub (40 stars)

About Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

rStar is a self-play mutual reasoning approach that significantly improves the reasoning capabilities of small language models (SLMs) without requiring fine-tuning or access to superior models. It decouples reasoning into a self-play mutual generation-discrimination process: a target SLM uses Monte Carlo Tree Search augmented with human-like reasoning actions to construct high-quality reasoning trajectories, and another SLM of similar capability acts as a discriminator to verify each trajectory. Only mutually consistent trajectories are considered correct. Experiments across five SLMs show substantial accuracy gains on benchmarks including GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA—for example, boosting GSM8K accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for Mistral-7B, and from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code is made available under an open-source license.

Key Features

Self-play mutual generation-discrimination process
Monte Carlo Tree Search (MCTS) augmented with human-like reasoning actions
No fine-tuning or superior model required
Uses two small language models of similar capability as generator and discriminator
Mutual consistency verification for reasoning trajectories
Demonstrated on multiple SLMs (LLaMA2-7B, Mistral-7B, LLaMA3-8B-Instruct, etc.)
Evaluated on diverse reasoning benchmarks (GSM8K, GSM-Hard, MATH, SVAMP, StrategyQA)
Open-source code release

Pros & Cons

Pros
  • Significantly improves reasoning accuracy of small LLMs without fine-tuning
  • Works with models of similar size; no need for a larger teacher model
  • Self-play mechanism leverages mutual consistency to filter correct reasoning paths
  • Applicable across multiple model architectures and scales
  • Demonstrates large absolute gains (e.g., +51.4% on LLaMA2-7B GSM8K)
  • Code is open-source, enabling reproduction and further research

Best For

Improving mathematical reasoning in small language modelsEnhancing commonsense and symbolic reasoning for SLMsBoosting performance on grade-school math problems (GSM8K, GSM-Hard, SVAMP)Solving challenging math problems (MATH)Answering strategy and reasoning questions (StrategyQA)

FAQ

How does rStar improve reasoning in small language models?
rStar uses a self-play mutual reasoning approach where one small language model generates reasoning trajectories via Monte Carlo Tree Search with human-like actions, and another small model discriminates their correctness. Only mutually consistent trajectories are retained, leading to higher accuracy without fine-tuning or larger models.