Unsafer in many turns: Benchmarking and defending multi-turn safety risks in tool-using agents

Analysis

Why This Paper Matters

As AI agents increasingly interact with tools and external systems over multiple turns, safety risks compound in ways that single-turn evaluations miss. This paper directly addresses a growing blind spot in AI safety research: the assumption that a model safe in one turn remains safe across a conversation. By demonstrating a 16% average increase in attack success rate (ASR) over multiple turns, the authors provide concrete evidence that multi-turn interactions are fundamentally more dangerous.

The introduction of MTAgentRisk fills a critical gap. Existing safety benchmarks (e.g., SafetyBench, AdvBench) focus on single-turn attacks, leaving tool-using agents—which are deployed in high-stakes environments like code execution, database queries, and API calls—without adequate evaluation. This paper's taxonomy of multi-turn risks also offers a structured way to think about attack surfaces that evolve over time.

Technical Contributions

• Multi-turn safety taxonomy: A structured classification of risks that emerge only across multiple interactions, such as incremental privilege escalation, context poisoning, and delayed payload delivery.
• MTAgentRisk benchmark: The first dedicated multi-turn safety benchmark for tool-using agents, covering diverse tool-use scenarios (e.g., file manipulation, web search, code execution).
• Empirical evaluation: Systematic measurement of ASR degradation across turns for multiple agent models, establishing a baseline for future defense research.

Results

The key quantitative finding is a 16% average increase in ASR when moving from single-turn to multi-turn evaluation. This metric is particularly striking because it aggregates across different agent architectures and tool-use domains, suggesting the vulnerability is general. The paper does not provide per-model breakdowns in the abstract, but the aggregate result is sufficient to motivate urgent attention to multi-turn safety.

Significance

This research has immediate implications for the deployment of AI agents in production. Many real-world agent systems (e.g., AutoGPT, ChatGPT plugins, coding assistants) operate over multiple turns, and current safety evaluations likely underestimate real-world risk. The benchmark provides a standardized way to measure and compare defenses, accelerating progress toward safer multi-turn agents. The taxonomy also offers a framework for red-teaming and safety auditing, which could become standard practice in agent development pipelines.