A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
AgentSpec introduces a customizable runtime enforcement framework for safe and reliable LLM agents, formalizing agents and enabling user-defined safety constraints.
Unknown
An agentic multimodal RAG framework for nuclear regulatory review achieves 69.0% accuracy, outperforming Unified Multimodal RAG by 6.5%.
Yuanhe Zhang, Weiliu Wang, Jie Ren, et al.
Proposes ILL, an inaudible red teaming method using low-frequency waveforms to attack LALMs, and DRG, a defense that detects distribution shifts and requests reacquisition, reducing accuracy by up to 67 points.
Asaf Yehudai, Lilach Eden, Alan Li, et al.
First comprehensive survey of evaluation methods for LLM-based agents, analyzing five perspectives and identifying gaps in cost-efficiency, safety, and robustness.
Josef Dai, Xuehai Pan, Ruiyang Sun, et al.
Safe RLHF decouples reward modeling and safety constraints to train language models that are both helpful and safe.
Yida Lu, Jiale Cheng, Zhexin Zhang, et al.
This paper introduces LongSafety, a benchmark and framework for evaluating the safety of large language models in long-context scenarios, revealing that safety risks increase with context length.
Unknown
This review explores mechanistic interpretability as a tool for AI safety, assessing its potential to prevent catastrophic outcomes.
Unknown
This paper argues that AI safety benchmarks may not actually measure safety progress due to the Bitter Lesson and scaling, suggesting safety metrics could be correlated with capabilities.
Unknown
A survey of practical methods for AI safety, covering inspection, understanding, and overcoming challenges in deploying deep neural networks in safety-critical applications.
Unknown
Introduces v0.5 of the MLCommons AI Safety Benchmark, a comprehensive suite for evaluating AI safety to drive innovation and better safety processes.
Unknown
This paper outlines a research agenda for AI safety, identifying five practical problems—accidents, misspecification, and other risks—that arise from poorly specified goals in machine learning systems.
Bandana Kaur
This paper defines the evidential ceiling of AI red-team evaluations, deriving a closed-form boundary that separates what benchmarks can and cannot prove about model safety.