A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Arian Hosseini, Alessandro Sordoni, Daniel Toyama, et al.
This paper reveals a significant reasoning gap in LLMs when solving compositional math problems, showing that performance on standard benchmarks masks systematic differences in reasoning abilities.
Siwei Wu, Z. Peng, Xinrun Du, et al.
This paper analyzes OpenAI's o1 model's reasoning patterns, comparing it with existing test-time compute methods on general reasoning benchmarks and summarizing six distinct reasoning patterns.
Zhuoshi Pan, Qizhi Pei, Yu Li, et al.
Introduces REST, a stress-testing framework that evaluates large reasoning models by presenting multiple problems simultaneously, revealing performance degradation and stronger discriminative power than single-question benchmarks.
Yizhe Chi, Wenyi Li, Deyao Hong, et al.
AI4AI-Bench benchmarks LLM agents on rewriting training algorithms across 10 repositories, finding best agents close only a fifth of the gap to optimal.
Zeyu Ren, Ling Yue, Ran Li, et al.
FlowEvo is a training-free framework where workflows and executable skills co-evolve at inference time, improving agent performance across benchmarks while reducing token usage.
Xinze Li, Yixin Cao, Yubo Ma, et al.
This paper compares Long Context and RAG for LLMs, finding LC generally outperforms RAG on QA benchmarks, with summarization-based retrieval matching LC, and highlights context relevance as an overlooked factor.
Bo Liu, Qiang Liu
Maglev is a recurrent Transformer architecture with fixed-size memory that generalizes sliding-window attention, trains with a memory consistency loss, and improves validation loss and downstream benchmarks.
Ning Li, Qiqiang Lin, Zheng Wu, et al.
ColorAgent is an OS agent that uses step-wise reinforcement learning and a multi-agent framework to achieve state-of-the-art success rates on AndroidWorld and AndroidLab benchmarks.
Corby Rosset, Pratyush Sharma, Andrew Zhao, et al.
The Universal Verifier improves computer use agent evaluation by combining rubric design, process/outcome rewards, cascading-error-free scoring, and divide-and-conquer context management, achieving human-level agreement and near-zero false positives.
Cheng Qian, Wenting Zhao, Liangwei Yang, et al.
This paper introduces test-time strong-to-weak capability transfer via inference-time harnesses, nearly doubling target model performance on Theory-of-Mind benchmarks without parameter updates.
Obed Junias, Maria Leonor Pacheco
Introduces a framework that decomposes compound answer options into atomic judgments and uses an operator-constrained ILP to compose calibrated scores, improving Macro-F1 on logical reasoning benchmarks.
Todor Ivanov, V. Penchev
This paper surveys AI benchmarks and datasets for evaluating large language models, providing a structured overview of existing resources and their applications.