Cooperate or collapse: Emergence of sustainable cooperation in a society of llm agents

Analysis

Why This Paper Matters

As large language models (LLMs) increasingly operate in multi-agent settings, understanding whether they can cooperate to manage shared resources is critical for deploying safe and sustainable AI systems. This paper introduces GOVSIM, a benchmark that simulates a society of LLM agents tasked with sustaining a common resource. The findings are sobering: most agents fail to achieve a sustainable equilibrium, with survival rates below 54%. This highlights a fundamental gap in current LLM capabilities for long-term cooperative behavior, which is essential for applications ranging from economic simulations to collaborative AI systems.

The significance extends beyond academic interest. Real-world multi-agent systems, such as those in supply chain management, energy grids, or autonomous vehicle coordination, require robust cooperation to avoid resource depletion or conflict. If LLM agents cannot sustain cooperation even in a controlled simulation, deploying them in critical infrastructure could lead to catastrophic failures. Thus, GOVSIM serves as an early warning and a diagnostic tool for the AI community.

Technical Contributions

• GOVSIM Benchmark: A novel simulation environment where LLM agents must collectively manage a shared resource, providing a standardized testbed for cooperative behavior.
• Empirical Evaluation: Systematic testing of multiple LLMs reveals that only the most powerful models achieve sustainable cooperation, with survival rates below 54% for all but the top performers.
• Ablation Studies: The paper identifies key factors that influence successful cooperation, such as agent capabilities and communication protocols, offering insights into what enables or hinders sustainable equilibria.
• Capability Threshold: Evidence that a certain level of LLM sophistication is necessary for cooperative success, suggesting a scaling law for social intelligence.

Results

The headline result is that the highest survival rate among tested LLM agents is below 54%, meaning that in most cases, the simulated society collapses due to resource depletion. Ablations show that successful multi-agent cooperation depends on the agents' ability to reason about others' actions and to make credible commitments. The most powerful LLMs, presumably those with advanced reasoning and theory-of-mind capabilities, are the only ones that approach sustainable outcomes. These findings are consistent across multiple runs, indicating robustness.

Significance

This work has profound implications for the development of multi-agent AI systems. It suggests that current LLMs, despite their impressive individual capabilities, lack the social intelligence needed for sustainable cooperation. This gap must be addressed before deploying LLM societies in real-world scenarios. GOVSIM provides a foundation for future research on enhancing cooperative abilities, potentially through improved training objectives, architectural innovations, or multi-agent reinforcement learning. Ultimately, this paper contributes to the broader goal of aligning AI systems with human values, where cooperation and sustainability are paramount.