Interpretable Contrastive Monte Carlo Tree Search Reasoning
FreeImproving LLM reasoning with interpretable MCTS
FreeFree tier
Inputs: textOutputs: text
About Interpretable Contrastive Monte Carlo Tree Search Reasoning
SC-MCTS* is a novel Monte Carlo Tree Search (MCTS) reasoning algorithm for Large Language Models (LLMs) that significantly improves both reasoning accuracy and speed. It introduces an interpretable reward model based on contrastive decoding, achieves an average speed improvement of 51.9% per node through speculative decoding, and enhances UCT node selection and backpropagation strategies. The algorithm outperforms o1-mini by an average of 17.4% on the Blocksworld multi-step reasoning dataset using Llama-3.1-70B. Code is available on GitHub.
Key Features
Interpretable reward model based on contrastive decoding
Speculative decoding for 51.9% average speed improvement per node
Improved UCT node selection and backpropagation strategies
Outperforms o1-mini by 17.4% on Blocksworld multi-step reasoning using Llama-3.1-70B
Open-source code available
Pros & Cons
Pros
- Significantly improves reasoning accuracy and speed over previous MCTS approaches
- Highly interpretable reward model due to contrastive decoding
- Achieves notable speed improvement via speculative decoding
- Outperforms a strong baseline (o1-mini) on a complex reasoning dataset
- Code is open-source, enabling reproduction and further research
Cons
- Still based on MCTS, which can be slower than simpler methods like Chain-of-Thought
- Primarily evaluated on one dataset (Blocksworld); generalizability to other tasks needs further testing
- As a research paper, it may not be immediately usable as a production-ready tool
Best For
Multi-step reasoning tasks for large language modelsPlanning and reasoning in domains like BlocksworldResearch on interpretability and efficiency of MCTS for LLMs
FAQ
What is SC-MCTS*?
SC-MCTS* is a novel Monte Carlo Tree Search reasoning algorithm for large language models, designed to improve both reasoning accuracy and speed through interpretable contrastive decoding, speculative decoding, and improved tree search strategies.
How does SC-MCTS* improve reasoning speed?
It uses speculative decoding to achieve an average speed improvement of 51.9% per node compared to previous MCTS methods.
What datasets was SC-MCTS* tested on?
The algorithm was tested on the Blocksworld multi-step reasoning dataset, where it outperformed o1-mini by an average of 17.4% using Llama-3.1-70B.
Is the code for SC-MCTS* available?
Yes, the code is open-source and available at the GitHub link provided in the arXiv paper.