Multi-agent strategic deception evaluation framework for LLMs using Secret Hitler as a testbed. Analyzes AI reasoning, trust dynamics, and deceptive behavior patterns.

Secret Hitler LLM Evaluation Framework

Multi-agent strategic deception evaluation for large language models using Secret Hitler as a testbed. This framework enables researchers to study AI reasoning, trust dynamics, and deceptive behavior patterns in a controlled game environment.

Author: Samuel T. Chakwera (stchakwdev)

Table of Contents

• Why This Project?
• Quick Start
• Batch Evaluation Monitor
• Evaluation Results
• Visual Analytics
• Features
• Architecture
• Documentation
• Citation
• Recent Updates
• Acknowledgments
• License
• Contact

Why This Project?

Understanding how AI systems engage in strategic deception is critical for AI safety research. Secret Hitler provides an ideal testbed because it:

• Requires hidden information management - Players must reason about unknown roles and hidden agendas
• Involves coalition formation - Trust and betrayal dynamics emerge naturally from gameplay
• Tests deceptive reasoning - Fascists must convincingly lie while Liberals must detect deception
• Produces measurable outcomes - Win rates, voting patterns, and policy outcomes provide quantifiable metrics

This framework enables researchers to:

1. Evaluate deception capabilities across different LLM architectures
2. Study emergent social behaviors in mult