Neura Market
knowledge-gpt logo

knowledge-gpt

Free

Extract knowledge from information sources.

FreeFree tier
Type
Open Source
Visit Website GitHub (40 stars)

About knowledge-gpt

Knowledge-gpt is an open-source Python library that extracts knowledge from diverse sources—including websites, PDFs, PowerPoint files (PPTX), documents (Docs), YouTube subtitles, and audio (via speech-to-text)—and uses OpenAI's GPT-3 (or other language models) to generate answers. The text is transformed into fixed-size vector embeddings using open-source (HF) or OpenAI models, and when a query is submitted, it is converted to a vector and compared against stored embeddings. The most relevant information is selected as prompt context for the language model. Answers can be saved to a database (e.g., MongoDB) for future reference. The library provides specialized extractors (WebScrapeExtractor, PDFExtractor, PowerpointExtractor, etc.) and is installed via 'pip install knowledgegpt'.

Key Features

Supports extraction from websites, PDFs, PPTX, Docs, YouTube subtitles, and audio (speech-to-text)
Uses OpenAI GPT-3 or other language models for answer generation
Text transformed into vector embeddings via open-source (HF) or OpenAI models
Indexes knowledge and enables Q&A sessions with stored information
Open-source Python library easily installable via pip
Provides specialized extractors: WebScrapeExtractor, PDFExtractor, PowerpointExtractor, BaseExtractor

Pros & Cons

Pros
  • Open-source and free (no licensing cost)
  • Supports a wide variety of input sources (web, PDF, PPTX, Docs, YouTube, audio)
  • Uses vector embeddings for relevance-based retrieval
  • Can index knowledge for future queries
  • Easy setup via pip with minimal dependencies
Cons
  • Requires OpenAI API key for GPT-3 access (cost may apply)
  • Requires downloading a language model (spaCy) for text parsing
  • Not a standalone application; requires Python environment and dependency management
  • Accuracy depends on the quality of the language model and embeddings used

Best For

Extracting knowledge from websites for Q&ACreating prompts from PDFs and documentsBuilding a knowledge base from multiple file typesAnswering questions based on YouTube video contentGenerating answers from local data sources

FAQ

What sources does knowledge-gpt support?
Knowledge-gpt supports websites, PDFs, PowerPoint files (PPTX), documents (Docs), YouTube subtitles, and audio (via speech-to-text).
How do I install knowledge-gpt?
Install via pip: 'pip install knowledgegpt'. Alternatively, clone the repository and run 'pip install -r requirements.txt' then 'pip install .'.
Does it require an API key?
Yes, you need an OpenAI API key. Set it in the configuration or environment before using the library.
What embedding extractors are available?
You can use either open-source (HF) or OpenAI models for generating embeddings. Specify with the 'embedding_extractor' parameter.
Can I use knowledge-gpt with multiple documents?
Yes, it indexes information from multiple sources and allows querying across all stored knowledge. Answers can be saved to a database like MongoDB.