A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
An empirical study showing that fine-tuned judge models are not a general substitute for GPT-4 in LLM evaluation, urging caution in real applications.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, et al.
This survey systematically addresses how to build reliable LLM-as-a-Judge systems, covering strategies to enhance reliability, evaluation methodologies, and a novel benchmark.
Yulai Zhao, Haolin Liu, Dian Yu, et al.
This paper reveals that LLM judges are vulnerable to reward hacking via simple 'master key' tokens, and proposes Master Reward Models to defend against such attacks.
Unknown
A generative PRM trained on small synthetic data outperforms discriminative PRMs and LLM-as-a-Judge baselines in step-by-step reasoning verification.
Unknown
A pipeline that generates system messages and aligned assistant responses for SFT datasets lacking them, using LLM-as-a-judge verification.
Unknown
Proposes an iterative self-training method using only synthetic data to improve LLM-as-a-Judge without human annotations.