A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
This paper proposes a multi-agent evaluation framework that aligns LLM-agent-based automated evaluation with multi-dimensional human evaluation for real-world text generation.
Unknown
Wildclawbench introduces a benchmark for evaluating frontier AI models on long-horizon, native-runtime agent tasks, showing that current models still struggle with such evaluations.
Unknown
This paper proposes a holistic agent leaderboard to standardize AI agent evaluation and address common pitfalls, shifting focus from benchmark-acing to real-world utility.
Unknown
This paper re-examines model editing under realistic deployment conditions, arguing that current evaluation methods overestimate real-world performance.
Unknown
This paper critically evaluates speculative decoding's real-world effectiveness, revealing that prior benchmarks overstate its speedups due to idealized conditions.
Kento Kawaharazuka, T. Matsushima, Andrew Gambardella, et al.
This paper reviews the application of foundation models in real-world robotics, categorizing approaches and discussing challenges and future directions.
Unknown
This paper introduces a method for learning symbolic world models from real-world observations to enable long-horizon planning in unseen environments.
Unknown
OSWorld-MCP is the first comprehensive and fair benchmark for assessing computer-use agents' tool invocation, GUI operation, and decision-making abilities in real-world scenarios.
Unknown
This paper provides a comprehensive guide to building agentic AI systems, covering frameworks, prompt engineering, and multi-agent orchestration for real-world applications.
Unknown
Introduces Vitabench, a benchmark for evaluating LLM agents on versatile interactive tasks in real-world applications, capturing complexity of handling extensive information.
Lizhi Yang, Junheng Li, Aaron D. Ames
PAC-MAN couples control-barrier safety with perception-aware training for whole-body humanoid dodgeball, achieving 95% real-world evasion with a fixed camera.
Unknown
Nemotron-flash introduces a hybrid small language model architecture that optimizes latency for efficient deployment in real-world applications.