LLama Cpp Agent

The llama-cpp-agent framework is a tool designed for simplified interactions with Large Language Models (LLMs). It provides an interface for chatting with LLMs, executing single and parallel function calls, generating structured output, performing retrieval augmented generation (RAG) with colbert reranking, and processing text using agentic chains (conversational, sequential, mapping). The framework uses guided sampling (grammars and JSON schema generation) to constrain model output, enabling even models not fine-tuned for JSON output or function calls to achieve these capabilities. It supports multiple providers including llama-cpp-python, llama.cpp server, TGI server, and vllm server. The tool is compatible with Python functions, Pydantic tools, LlamaIndex tools, and OpenAI tool schemas. Note: The project is no longer actively maintained; users are directed to ToolAgents or other Python agentic frameworks.