GitHub logo

GitHub

Free

Open Source, Language Agnostic Mutation Testing

FreeFree tier
Type
Open Source
Company
codeintegrity-ai

About GitHub

Mutahunter is an open-source, language-agnostic mutation testing tool that leverages large language models (LLMs) to automatically generate and evaluate code mutants. It helps developers assess the quality and robustness of their test suites by introducing small code changes (mutants) and checking whether existing tests can catch them. Mutahunter supports any programming language with a test command (e.g., Maven for Java) and integrates seamlessly into CI/CD pipelines. It provides detailed mutation coverage metrics, including percentage, total mutants, survived, killed, timeouts, and compile errors, along with cost tracking for LLM API usage. The tool is easily installed via pip and requires Python 3.11+.

Key Features

LLM-based mutant generation using models like GPT-4o-mini
Language agnostic; works with any language by providing a test command
Detailed mutation coverage metrics (percentage, survived, killed, timeout, compile errors)
Cost tracking for LLM API usage (example: $0.00060 per run)
Easy installation via pip (Python 3.11+ required)
Integrates with existing test frameworks (e.g., Maven for Java)
Open source under AGPL-3.0 license

Pros & Cons

Pros
  • Open source and free to use (AGPL-3.0 license)
  • Language agnostic; supports any programming language with a test command
  • Uses LLMs to generate intelligent, context-aware mutants
  • Cost-effective; minimal API costs for typical runs
  • Provides clear, actionable mutation coverage statistics
  • Simple setup with pip and environment variables
Cons
  • Requires an OpenAI API key (or other LLM provider) for mutation generation
  • LLM API costs can accumulate for large codebases or frequent runs
  • Currently no official releases or versioned packages (only install from GitHub)
  • Documentation is limited to the README; no extensive guides or examples beyond Java
  • May have compile errors in generated mutants that skew coverage results

Best For

Assessing the effectiveness of unit test suitesIdentifying weak or missing tests in software projectsIntegrating mutation testing into CI/CD pipelines for continuous quality assuranceTeaching and learning about test coverage and mutation testingImproving code reliability in open-source and enterprise projects

FAQ

How do I install Mutahunter?
Install via pip using the command: pip install https://github.com/codeintegrity-ai/mutahunter. Python 3.11+ is required.
Which LLM models are supported?
Mutahunter works with models like GPT-4o-mini. You set the model via the --model flag (e.g., --model gpt-4o-mini). An OpenAI API key is required.
What programming languages are supported?
Mutahunter is language agnostic. It works with any language that can be tested via a command-line test command. An example for Java (Maven) is provided.
What output does Mutahunter provide?
It outputs a mutation coverage report including overall coverage percentage, total mutants, survived, killed, timeout, compile error mutants, and total cost in USD.
Is Mutahunter free?
The tool itself is open source and free to use under the AGPL-3.0 license. However, you will need your own OpenAI API key, which may incur charges based on usage.