JudgeLRM: Large Reasoning Models as a Judge

Free

Large Reasoning Models as a Judge

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About JudgeLRM: Large Reasoning Models as a Judge

JudgeLRM is a family of judgment-oriented large language models developed through reinforcement learning (RL) with judge-wise, outcome-driven rewards to activate reasoning capabilities for evaluation tasks. Targeting the limitations of supervised fine-tuning (SFT) in reasoning-intensive domains, JudgeLRM consistently outperforms SFT-tuned baselines of the same size and surpasses state-of-the-art reasoning models: JudgeLRM-3B/4B exceeds GPT-4, while JudgeLRM-7B/8B/14B outperforms DeepSeek-R1 by over 2% in F1 score, with particularly strong gains on reasoning-heavy tasks. The approach underscores the value of RL in unlocking reasoning-aligned LLM judges, enabling verification of evidence, error identification, and justification of decisions beyond surface-level scoring.

Key Features

Trained using reinforcement learning with judge-wise, outcome-driven rewards

Activates deep reasoning capabilities for evaluation tasks

Verifies evidence, identifies errors, and justifies decisions beyond surface-level scoring

Available in 3B, 4B, 7B, 8B, and 14B parameter sizes

Consistently outperforms SFT-tuned baselines of the same size

3B/4B models surpass GPT-4; 7B/8B/14B models outperform DeepSeek-R1 by over 2% F1

Strong gains on reasoning-heavy tasks

Pros & Cons

Pros

Consistently outperforms SFT-tuned baselines and larger reasoning models
Small models (3B/4B) achieve performance exceeding GPT-4
Reinforcement learning approach effectively unlocks reasoning in judges
Open-source availability enables community experimentation and deployment

Cons

Focused specifically on judgment and evaluation tasks, not general-purpose
Still a research preprint; production readiness may require further validation
Performance gains most pronounced on reasoning-heavy samples; may not benefit simpler evaluations
Requires RL training infrastructure and expertise to reproduce or adapt

Best For

Automated evaluation of LLM outputsLarge-scale assessment tasks requiring complex reasoningError detection and evidence verification in AI-generated contentScalable alternative to human annotation for judgment tasksBenchmarking and comparing model performance on reasoning-heavy evaluations

FAQ

What is JudgeLRM?

JudgeLRM is a family of judgment-oriented large language models trained using reinforcement learning with judge-wise, outcome-driven rewards to activate reasoning capabilities for evaluation tasks.

How does JudgeLRM compare to other LLM judges?

JudgeLRM consistently outperforms supervised fine-tuning (SFT) baselines of the same size and exceeds state-of-the-art reasoning models: the 3B/4B variants surpass GPT-4, while the 7B/8B/14B variants outperform DeepSeek-R1 by over 2% in F1 score, particularly on reasoning-heavy tasks.

What sizes are available?

JudgeLRM is available in 3B, 4B, 7B, 8B, and 14B parameter configurations.

What makes JudgeLRM different from SFT-based evaluators?

JudgeLRM uses reinforcement learning (RL) instead of supervised fine-tuning (SFT), which the authors show yields stronger performance on reasoning-demanding evaluation tasks where SFT gains are limited.