Biomedical RAG Chat Pipeline: End-to-End Flow

Overview

This pipeline enables Retrieval-Augmented Generation (RAG) for biomedical question answering, combining document retrieval (Qdrant) and chat memory (PostgreSQL/pgvector) with a state-of-the-art LLM (Groq Llama 3.3-70b). The system supports multi-turn chat, session-based memory, and scalable document ingestion.

---

Step-by-Step Flow

1. Google Drive Download

   • PDFs are downloaded from a specified Google Drive folder.
   • File: src/data_access/google_drive_access.py

2. PDF Parsing

   • PDFs are parsed into raw text.
   • File: src/data_access/pdf_parser.py

3. Document Chunking

   • Raw text is split into manageable chunks for embedding.
   • File: src/data_access/document_chunker.py

4. Embedding Generation & Storage

   • Each chunk is embedded using PubMedBERT.
   • Embeddings are stored in Qdrant (vector DB) under a session/collection name.
   • File: src/data_access/embeddings_manager.py

5. Session Management

   • All operations are session-based for isolation and multi-user support.
   • File: src/deep_recall/session_manager.py

6. Chat Memory Storage

   • User and assistant messages are embedded and stored in PostgreSQL with pgvector.
   • File: src/deep_recall/memory/memory_store.py

7. Dual Retrieval (RAG)

   • On each user query:
     • Document Retrieval: Top-K relevant chunks are retrieved from Qdrant.
     • Chat Retrieval: Top-K relevant chat messages are retrieved from PostgreSQL.
   • Files:
     • Document: src/data_access/embeddings_manager.py
     • Chat: src/deep_recall/memory/memory_store.py

8. Prompt Assembly

   • Retrieved document and chat context are combined into a single prompt.
   • File: src/llm/groq_llama.py (assemble_prompt)

9. LLM Response Generation

   • The prompt is sent to Groq's Llama 3.3-70b model via API.
   • The assistant's response is returned and stored in chat memory.
   • File: src/llm/groq_llama.py

10. Chat Loop

    • The user can continue asking questions; each turn updates chat memory and context.
    • File: src/llm/test_pipeline_groq.py

---

Flow Diagram

flowchart TD
    A[Download PDFs from Google Drive] --> B[Parse PDFs to Text]
    B --> C[Chunk Documents]
    C --> D[Generate Embeddings]
    D --> E[Store in Qdrant (Document Memory)]
    subgraph Chat Session
        F[User Query]
        F --> G[Store User Message in Chat Memory (Postgres/pgvector)]
        G --> H[Retrieve Top-K Docs from Qdrant]
        G --> I[Retrieve Top-K Chat Messages]
        H & I --> J[Assemble RAG Prompt]
        J --> K[Groq Llama 3.3-70b API]
        K --> L[Assistant Response]
        L --> M[Store Assistant Message in Chat Memory]
        L --> F
    end

---

How Document and Chat Memory Work Together

• Document Memory (Qdrant):
  • Stores vector embeddings of all document chunks.
  • Enables fast, semantic retrieval of factual biomedical knowledge.
• Chat Memory (PostgreSQL/pgvector):
  • Stores embeddings of all user and assistant messages per session.
  • Enables retrieval of relevant chat history for context continuity.
• Combined Retrieval:
  • Both memories are queried for each user input.
  • The LLM receives both factual and conversational context for optimal answers.

---

Chat Loop Summary

• Each user message and assistant response is stored in chat memory.
• Retrieval always includes the latest chat history and relevant documents.
• The loop continues until the user types exit or break.
• Only the user's question and the assistant's answer are printed for clarity.

---

Main Files/Modules by Step

Step	File/Module
Google Drive Download	google_drive_access.py
PDF Parsing	pdf_parser.py
Document Chunking	document_chunker.py
Embedding & Qdrant Storage	embeddings_manager.py
Session Management	session_manager.py
Chat Memory (Postgres)	memory_store.py
Document Retrieval	embeddings_manager.py
Chat Retrieval	memory_store.py
Prompt Assembly & LLM	groq_llama.py
Chat Loop	test_pipeline_groq.py

---

Summary

This pipeline enables robust, session-based biomedical RAG chat with dual memory retrieval and state-of-the-art LLM responses. It is modular, scalable, and ready for research or production use.