Neura Market
PrivateGPT logo

PrivateGPT

Free

Open-source API layer for private AI on local models

FreeFree tier
Inputs: text, fileOutputs: text, code
Type
Open Source
Company
Zylon AI
Visit Website GitHub (38 stars)

About PrivateGPT

PrivateGPT is an open-source API layer that turns local models into production AI applications. It provides a comprehensive set of higher-level building blocks following the Claude API model, enabling developers to build private AI products without depending on cloud APIs. Key capabilities include a standard messages API with streaming, async, and token counting; file and artifact ingestion; retrieval with citations and agentic RAG; built-in tools such as web search, web fetch, and code execution; custom tools and MCP connectors; structured access to databases and CSVs; and embeddings and orchestration. PrivateGPT works with any OpenAI-compatible inference server (e.g., Ollama, llama.cpp, vLLM) and includes a built-in workbench UI for testing and demos. It is production-tested and powers Zylon, an on-premise AI platform used by enterprises globally.

Key Features

Standard messages API with streaming, async, and token counting
File and artifact ingestion
Retrieval with citations and agentic RAG
Built-in tools: web search, web fetch, code execution
Custom tools and MCP connectors
Structured access to databases and CSVs
Embeddings and orchestration
Built-in workbench UI available at /ui
Compatible with any OpenAI-compatible inference server

Pros & Cons

Pros
  • Fully open-source and self-hostable
  • Works with a wide range of local inference servers (Ollama, vLLM, etc.)
  • Comprehensive API covering messaging, RAG, tools, and data access
  • Production-tested and powers an enterprise AI platform
  • Built-in UI simplifies development and testing
Cons
  • Requires a separate OpenAI-compatible LLM server to be running
  • Setup may be complex for non-technical users needing model configuration
  • Does not include its own models; relies on external inference servers

Best For

Building private AI applications and agents without cloud dependenciesEnterprise document Q&A and knowledge retrieval with citationsInternal code assistance with code execution toolsData analysis via structured database and CSV accessPrototyping and testing AI workflows with the built-in UI

FAQ

Does PrivateGPT run its own AI models?
No, PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server (e.g., Ollama, llama.cpp, vLLM) via the OPENAI_API_BASE environment variable.
What models are compatible with PrivateGPT?
Any model served by an inference server that implements the /v1/chat/completions and /v1/models endpoints is compatible.
Is there a user interface for testing?
Yes, PrivateGPT ships a built-in workbench UI available at /ui for sending messages, selecting models, and uploading documents.
How do I install PrivateGPT?
Installation instructions are available for macOS (Homebrew), Linux (uv tool), and Windows (PowerShell). A Docker quickstart is also provided.