AI agents - performance comparison using major LLMs.

Overview

Experiment on AI agent performance using versionhq and pydantic.

Visit:

• PyPI
• Github
• Docs
• Process (Medium story)

Key Features

Generate multi-agent systems based on the task complexity, execute tasks, and evaluate output based on the given criteria.

Agents are model-agnostic, and can handle and share RAG tools, knowledge, memory, and callbacks among other agents.

Quick Start

1. Install uv package manager:

   For MacOS:

   brew install uv
   

   For Ubuntu/Debian:

   sudo apt-get install uv
   

2. Install dependencies:

   uv venv
   source .venv/bin/activate
   uv pip install -r requirements.txt
   

• In case of AssertionError/module mismatch, run Python version control using .pyenv

  pyenv install 3.12.8
  pyenv global 3.12.8  (optional: `pyenv global system` to get back to the system default ver.)
  uv python pin 3.12.8
  echo 3.12.8 > .python-version
  

3. Set up environment variables: Create .env file in the project root and add the following:

   OPENAI_API_KEY=your-openai-api-key
   GEMINI_API_KEY=your-gemini-api-key
   OPENROUTER_API_KEY=your-openrouter-api-key
   

4. Run:

   uv run main.py
   

Customizing

• To add or refine an agent, use src/agents.py.

• To add or refine a task, use src/tasks.py.

Results

(Feb 3, 2025)

Trouble Shooting

Common issues and solutions:

• API key errors: Ensure all API keys in the .env file are correct and up to date. Make sure to add load_dotenv()