**garak**
Free
About **garak**
garak is a free, open-source command-line tool developed by NVIDIA for probing and assessing vulnerabilities in large language models (LLMs). It functions as a red-teaming assessment kit, similar to security tools like nmap or Metasploit Framework, but focused on LLMs. garak checks if an LLM can be made to fail in undesirable ways by probing for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses. It combines static, dynamic, and adaptive probes and supports a wide range of LLM backends including Hugging Face, OpenAI, AWS Bedrock, Replicate, LiteLLM, and GGUF models. The tool is maintained by NVIDIA and has an active community with a Discord server and Twitter presence.
Key Features
Pros & Cons
- Free and open-source with no usage restrictions
- Comprehensive probe library covering many known LLM weaknesses
- Supports a wide variety of LLM backends and interfaces
- Active development and community contribution
- Designed for both static and adaptive probing, similar to professional security tools
- Command-line only, lacking a graphical user interface
- Requires technical expertise to set up and configure (e.g., API keys, model names)
- Documentation may be spread across multiple sources (GitHub, docs.garak.ai)
- Some probes may require significant computational resources depending on the model size