A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
M. G. Bellemare, Y. Naddaf, J. Veness, et al.
Introduces the Arcade Learning Environment (ALE), a platform and methodology for evaluating general, domain-independent AI agents across hundreds of Atari 2600 games.
Hideyuki Takagi
A comprehensive survey of interactive evolutionary computation (IEC), covering applications and interface research to reduce human fatigue, based on about 250 papers.
Micah Hodosh, Peter Young, Julia Hockenmaier
Frames image description as a ranking task, introducing a benchmark of 8,000 images with five captions each and showing that ranking-based evaluation can be automated.
Catalin Ionescu, Dragos Papava, Vlad Olaru, et al.
Introduces Human3.6M, a large-scale dataset of 3.6 million accurate 3D human poses with synchronized images, motion capture, and depth data, along with statistical models and evaluation baselines for 3D human sensing.
Aaron Chase, Amoreena Most, Andrea Sikora, et al.
This study evaluates GPT-4, Gemini 1.5, and Claude 3 on identifying drug-drug interactions and generating clinical recommendations, finding high identification rates but variable recommendation quality and potential harm.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, et al.
This survey systematically addresses how to build reliable LLM-as-a-Judge systems, covering strategies to enhance reliability, evaluation methodologies, and a novel benchmark.
Asaf Yehudai, Lilach Eden, Alan Li, et al.
First comprehensive survey of evaluation methods for LLM-based agents, analyzing five perspectives and identifying gaps in cost-efficiency, safety, and robustness.
Corby Rosset, Pratyush Sharma, Andrew Zhao, et al.
The Universal Verifier improves computer use agent evaluation by combining rubric design, process/outcome rewards, cascading-error-free scoring, and divide-and-conquer context management, achieving human-level agreement and near-zero false positives.
Todor Ivanov, V. Penchev
This paper surveys AI benchmarks and datasets for evaluating large language models, providing a structured overview of existing resources and their applications.
Unknown
This paper identifies common tropes in LLM evaluation practices and validates them through two industry case studies, highlighting the need for more rigorous evaluation methods.
Ali Borji, Ming-Ming Cheng, Qibin Hou, et al.
A comprehensive survey of salient object detection covering 228 publications, including roots, core techniques, datasets, evaluation metrics, and future directions.
Yijia Xiao, Wanjia Zhao, Junkai Zhang, et al.
This paper provides the first comprehensive overview of Protein LLMs, covering architectures, training datasets, evaluation metrics, and applications, with a structured taxonomy from over 100 articles.