LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning

Analysis

Why This Paper Matters

Selecting the correct answer from a pool of candidate reasoning chains is a critical component of test-time scaling for large language models (LLMs). Current methods, such as self-consistency and process reward models (PRMs), each have drawbacks: self-consistency can propagate errors from individual models, while PRMs require additional training and may not generalize. This paper introduces a novel approach—using a jury of multiple LLMs to reach a consensus—that leverages model diversity to improve selection accuracy. By demonstrating that cross-model consensus can outperform PRMs, the work challenges the dominance of single-model selection strategies and opens new avenues for enhancing LLM reasoning without additional training.

The significance lies in its potential to make test-time scaling more robust and accessible. As LLMs are increasingly deployed in high-stakes reasoning tasks, reliable answer selection becomes paramount. This paper's jury approach offers a simple yet effective alternative that could be integrated into existing pipelines, especially when multiple models are already available. It also highlights the value of ensemble methods in the LLM era, reminiscent of classical ensemble learning but adapted to the unique challenges of reasoning chains.

Technical Contributions

• Cross-Model Consensus Mechanism: The paper formalizes a jury-based selection where multiple LLMs independently evaluate candidate reasoning chains, and the most agreed-upon answer is chosen.
• Comparison with PRMs: It provides empirical evidence that this consensus approach can surpass PRMs, which are specialized models trained to score reasoning steps.
• Addressing Self-Consistency Limitations: By using different models, the method mitigates the error inheritance problem of self-consistency, where a single model's biases affect all samples.
• Practical Framework: The approach is model-agnostic and can be applied to any set of LLMs, making it broadly applicable.

Results

The abstract states that cross-model consensus outperforms process reward models for LLM reasoning. However, specific metrics, datasets, and quantitative comparisons are not provided in the abstract. The paper likely presents experiments on standard reasoning benchmarks (e.g., math or logic problems) where the jury method achieves higher accuracy than PRM-based selection. The improvement is attributed to the diversity of models, which reduces correlated errors. Without concrete numbers, the exact margin of improvement remains unclear, but the claim suggests a meaningful advantage.

Significance

This work has the potential to influence how practitioners approach test-time scaling. Instead of investing in training PRMs or relying on a single model's self-consistency, they can leverage an ensemble of existing models. This could lead to more reliable and cost-effective reasoning systems. Furthermore, it encourages further research into ensemble methods for LLMs, including optimal jury composition and voting schemes. As the field moves toward multi-agent and multi-model systems, this paper provides a timely contribution that bridges ensemble learning and LLM reasoning, potentially paving the way for more robust AI decision-making.