Solving math word problems with processand outcome-based feedback

Free

Comparing process- and outcome-based feedback for math reasoning in language models

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (40 stars)

About Solving math word problems with processand outcome-based feedback

This paper presents a comprehensive comparison between process-based and outcome-based supervision approaches for training language models to solve math word problems (GSM8K). The study finds that pure outcome-based supervision achieves similar final-answer error rates with less label supervision, but correct reasoning steps require process-based supervision or learned reward models that emulate process-based feedback. The authors improve previous state-of-the-art results, reducing final-answer error from 16.8% to 12.7% and reasoning error among final-answer-correct solutions from 14.0% to 3.4%. The work highlights the trade-offs between supervision types for reasoning tasks and has implications for education and real-world applications where reasoning accuracy matters.

Key Features

Comprehensive comparison of process-based vs outcome-based supervision on GSM8K

Pure outcome-based supervision reduces final-answer error rates with less label supervision

Process-based supervision necessary for correct reasoning steps

Learned reward models that emulate process-based feedback improve reasoning accuracy

Achieves 12.7% final-answer error and 3.4% reasoning error among correct solutions

Pros & Cons

Pros

Outcome-based supervision requires fewer labels while maintaining similar final-answer accuracy
Process-based supervision significantly reduces reasoning errors
Comprehensive comparison on a standard benchmark (GSM8K)
Improves upon previous best results on GSM8K

Cons

Process-based supervision requires more detailed labels (reasoning steps)
Learned reward models may not fully capture process-based feedback quality
Results are specific to math word problems; generalizability to other reasoning tasks is not explored

Best For

Solving math word problemsTraining language models on reasoning tasksEducational applications where reasoning accuracy is criticalEvaluating supervision strategies for chain-of-thought reasoning

FAQ

What is the main contribution of this paper?

The paper presents the first comprehensive comparison between process- and outcome-based supervision approaches for training language models on a natural language reasoning task (GSM8K). It shows trade-offs between the two methods and improves state-of-the-art results.

What dataset was used for the experiments?

The experiments were conducted on the GSM8K dataset, a collection of grade-school math word problems.

What were the key findings?

Pure outcome-based supervision yields similar final-answer error rates with less label supervision, but process-based supervision or learned reward models that emulate process-based feedback are necessary for achieving correct reasoning steps.