GitHub

LocalGPT is a fully private, on-premise Document Intelligence platform that enables users to chat with their documents using state-of-the-art AI models locally. It goes beyond traditional RAG tools by featuring a hybrid search engine that blends semantic similarity, keyword matching, and Late Chunking for long-context precision. A smart router automatically selects between RAG and direct LLM answering for each query, while contextual enrichment and sentence-level Context Pruning ensure only the most relevant content is surfaced. An independent verification pass adds an extra layer of accuracy. The architecture is modular and lightweight, with a pure-Python core and minimal dependencies, making it simple to deploy, run, and maintain on any infrastructure. LocalGPT supports multiple hardware platforms (GPU, CPU, HPU, MPS) and can integrate a variety of open-source models via Ollama. It handles multiple document formats including PDF, DOCX, TXT, and Markdown, and offers features like chat history, reusable LLMs, and an API for building RAG applications.