Multi-agent deep reinforcement learning: a survey

Analysis

Why This Paper Matters

Multi-agent reinforcement learning (MARL) is critical for real-world applications like autonomous driving, robotics, and game AI, where multiple agents interact. This survey arrives at a time when deep RL has achieved remarkable single-agent successes, but multi-agent settings introduce complexities such as non-stationarity, scalability, and coordination. By systematically organizing the literature, the paper helps researchers navigate a fragmented field and identify promising directions.

The paper's emphasis on training schemes (e.g., centralized training with decentralized execution) and emergent behaviors (cooperative, competitive, mixed) provides a clear framework for understanding how different approaches address multi-agent challenges. This is especially valuable for practitioners deciding which paradigm to adopt for their specific problem.

Technical Contributions

• Taxonomy of training schemes: Distinguishes between centralized, decentralized, and hybrid approaches, clarifying trade-offs in communication, scalability, and coordination.
• Behavioral pattern analysis: Categorizes emergent behaviors in cooperative, competitive, and mixed settings, linking them to underlying algorithms.
• Challenge enumeration: Lists key multi-agent challenges (non-stationarity, credit assignment, scalability) and reviews methods like parameter sharing, attention mechanisms, and communication protocols.
• Future directions: Identifies open problems such as transfer learning, safety, and human-agent interaction.

Results

As a survey, the paper does not present new experimental results. Its value lies in synthesizing over 900 citations to map the current landscape. It highlights that centralized training with decentralized execution (CTDE) is a dominant paradigm, and that value decomposition methods (e.g., VDN, QMIX) have shown strong performance in cooperative tasks. The survey notes that competitive and mixed scenarios remain less explored.

Significance

This survey serves as a foundational reference for the MARL community, consolidating knowledge from a rapidly expanding field. It enables researchers to quickly understand the state of the art, identify gaps, and choose appropriate methods. By outlining challenges and trends, it helps steer future research toward impactful problems, such as scaling to many agents and ensuring robustness in real-world deployments.