Thought-Level Beam Search for Reasoning

Analysis

Why This Paper Matters

Test-time compute scaling has become a primary driver of performance in large reasoning models (LRMs), but current approaches are extremely inefficient. The paper identifies a critical dichotomy: traditional parallel sampling treats reasoning traces independently, leading to severe memory bottlenecks and wasted compute on unpromising paths, while subtractive pruning (e.g., best-of-n or rejection sampling) starves hardware and fails to actively shift the output distribution. This inefficiency shifts the question from 'how much compute' to 'where to allocate it.'

Gambit directly addresses this by formalizing test-time reasoning as a constrained compute allocation problem over partial trajectories. By performing thought-level beam search, it dynamically concentrates compute onto the most promising reasoning traces, maintaining high hardware utilization. This is a significant step forward because it offers a principled way to allocate compute adaptively, rather than relying on static sampling or pruning strategies.

Technical Contributions

• Formalization: The paper formalizes test-time reasoning as a constrained compute allocation problem, providing a clear framework for comparing different inference strategies.
• Thought-Level Beam Search: Gambit executes beam search at the thought level, pruning unpromising trajectories and branching from high-quality prefixes. This is a novel approach that goes beyond token-level or trajectory-level methods.
• Light-Weight Scorer: Gambit uses a light-weight scorer that probes hidden states to rank partial trajectories, enabling efficient and accurate pruning without heavy overhead.
• Hardware Utilization: By immediately branching from high-quality prefixes, Gambit maintains continuous high hardware utilization, avoiding the starvation seen in subtractive pruning.
• Dynamic Compute Allocation: The algorithm actively shifts compute to the most promising reasoning traces, which is a key innovation over static allocation.

Results

The paper reports extensive evaluations across multiple models and benchmarks. Under identical hardware constraints, Gambit strictly dominates existing baselines. Specifically, it achieves up to +6.7% absolute accuracy gain on HMMT-24 and +3.3% on AIME-25 over pruning baselines. It also delivers >2x higher throughput on trace completion and reduces total token consumption by up to 68.5% relative to standard parallel sampling. These results are compelling because they show improvements in both accuracy and efficiency, which are often trade-offs in inference algorithms.

Significance

Gambit has the potential to significantly impact the deployment of large reasoning models by making test-time compute scaling more efficient. The ability to achieve higher accuracy with less compute and fewer tokens could reduce costs and latency in real-world applications. Moreover, the formalization of compute allocation opens up new research directions for adaptive inference algorithms. As LRMs become more prevalent, methods like Gambit will be crucial for making them practical and sustainable.