Secret_H_Evals

# Secret Hitler LLM Evaluation Framework

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
[![GitHub stars](https://img.shields.io/github/stars/stchakwdev/Secret_H_Evals?style=social)](https://github.com/stchakwdev/Secret_H_Evals)

Multi-agent strategic deception evaluation for large language models using Secret Hitler as a testbed. This framework enables researchers to study AI reasoning, trust dynamics, and deceptive behavior patterns in a controlled game environment.

**Author**: Samuel T. Chakwera ([stchakwdev](https://github.com/stchakwdev))

---

## Table of Contents

- [Why This Project?](#why-this-project)
- [Quick Start](#quick-start)
- [Batch Evaluation Monitor](#batch-evaluation-monitor)
- [Evaluation Results](#evaluation-results-300-games)
- [Visual Analytics](#visual-analytics)
- [Features](#features)
- [Architecture](#architecture)
- [Documentation](#documentation)
- [Citation](#citation)
- [Recent Updates](#recent-updates)
- [Acknowledgments](#acknowledgments)
- [License](#license)
- [Contact](#contact)

---

## Why This Project?

Understanding how AI systems engage in strategic deception is critical for AI safety research. Secret Hitler provides an ideal testbed because it:

- **Requires hidden information management** - Players must reason about unknown roles and hidden agendas
- **Involves coalition formation** - Trust and betrayal dynamics emerge naturally from gameplay
- **Tests deceptive reasoning** - Fascists must convincingly lie while Liberals must detect deception
- **Produces measurable outcomes** - Win rates, voting patterns, and policy outcomes provide quantifiable metrics

This framework enables researchers to:
1. **Evaluate deception capabilities** across different LLM architectures
2. **Study emergent social behaviors** in mult