WARP.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

Project Overview

RAG Troubleshooter is a production-ready RAG service for medical device troubleshooting with vendor-neutral LLM support. It provides hallucination-safe, explainable troubleshooting guidance by ingesting service manual PDFs into a vector database and generating grounded answers via Claude, OpenAI, or Gemini.

Development Commands

Environment Setup

# Install dependencies
poetry install

# Or with pip
pip install -e .

# Configure environment variables
Copy-Item .env.example .env
# Then edit .env with your API keys

Running the Server

# Run with Poetry (recommended for development with hot reload)
poetry run python app/main.py

# Or with uvicorn directly
poetry run uvicorn app.main:app --reload

# Production mode (no reload)
poetry run uvicorn app.main:app --host 0.0.0.0 --port 8000

Server endpoints:

• Demo UI: http://localhost:8000
• API Docs: http://localhost:8000/docs
• Health Check: http://localhost:8000/api/

Document Ingestion

# Run the ingestion script to load device manuals
poetry run python ingest_devices.py

# Or via API (requires running server)
curl -X POST http://localhost:8000/api/ingest `
  -H "Content-Type: application/json" `
  -H "X-API-Key: dev-key-123" `
  -d '{"device_name": "Bair Hugger Model 775", "filename": "bair_hugger_manual.pdf"}'

Testing

# Run tests (when implemented)
poetry run pytest

# Run tests with coverage
poetry run pytest --cov=app --cov-report=html

Code Quality

# Format code (line length: 100)
poetry run black app/

# Lint code
poetry run ruff app/

# Type check
poetry run mypy app/

Architecture Overview

Core Design Principles

1. Pluggable Components: Abstract base classes enable easy swapping of vector stores and LLM providers
2. Hallucination Prevention: Strict prompts, low temperature (0.1), required citations, "No grounded steps found" fallback
3. Async Throughout: All I/O operations are async for scalability
4. Stateless API: Easy horizontal scaling

Key Architectural Patterns

1. Provider Abstraction Pattern

Both vector stores and LLM providers use abstract base classes to enable pluggability:

• Vector Store: VectorStore (base) → LocalVectorStore (FAISS implementation)

  • Easy to add PineconeVectorStore, WeaviateVectorStore, etc.
  • All implement: add_documents(), search(), delete_by_metadata(), get_stats()

• LLM Providers: LLMProvider (base) → ClaudeProvider, OpenAIProvider, GeminiProvider

  • Unified interface: generate() method returns standardized LLMResponse
  • Configuration via LLMConfig dataclass

2. Service Layer Architecture

Three main services coordinate the RAG pipeline:

• IngestionService: PDF → chunks → embeddings → vector store

  • Handles PDF parsing, text chunking with overlap, batch embedding generation
  • Chunking is sentence-boundary aware (breaks at . or \n when possible)

• RetrievalService: Query → embeddings → semantic search → re-ranking → context

  • Two-stage retrieval: semantic search (FAISS) + lightweight keyword re-ranking
  • Re-ranking formula: 0.7 * semantic_score + 0.2 * keyword_overlap + 0.1 * term_density

• RAGService: Query + context → LLM → grounded answer

  • Enforces hallucination-averse system prompt
  • Supports multi-provider comparison (same context, different LLMs)

3. Hallucination Prevention Strategy

The system is designed to prevent hallucinations through multiple mechanisms:

• System Prompt: Explicitly instructs models to use ONLY provided context or respond with "No grounded steps found"
• Low Temperature: Set to 0.1 (configurable) to minimize creativity
• Token Limit: 500 tokens max (configurable) forces concise answers
• Citation Requirement: Every answer must cite sources with [Source N] notation
• Context Validation: Returns early with "No grounded steps found" if retrieval yields no results

4. Retrieval Pipeline

The retrieval process follows this flow:

1. Chunking (Ingestion):

   • 512-character chunks with 50-character overlap (configurable)
   • Sentence-boundary aware splitting to avoid mid-sentence breaks
   • Metadata attached: device, page, chunk_index, source_file

2. Semantic Search:

   • Query embedded using all-MiniLM-L6-v2 (configurable)
   • FAISS cosine similarity search
   • Optional device-based metadata filtering
   • Retrieves 2x requested chunks for re-ranking

3. Re-ranking:

   • Combines semantic score with keyword overlap and term density
   • Returns top-k chunks with highest combined scores

4. Context Formatting:

   • Formats retrieved chunks as: [Source N] (Page X, Device):\n{text}
   • Builds citation list with chunk IDs, pages, relevance scores

Data Flow

Query → RetrievalService → VectorStore → [chunks] 
     → RAGService → LLMProvider → Answer + Citations

For multi-provider comparison:

Query → RetrievalService (once) → [shared context]
     → RAGService → [Claude, OpenAI, Gemini] → [Answer1, Answer2, Answer3]

Configuration System

Configuration uses Pydantic Settings with .env file support:

• API Keys: ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY
• Vector Store: VECTOR_STORE_TYPE, VECTOR_STORE_PATH
• Embeddings: EMBEDDING_MODEL
• LLM: DEFAULT_LLM_PROVIDER, LLM_TEMPERATURE, LLM_MAX_TOKENS
• Retrieval: TOP_K_CHUNKS, CHUNK_SIZE, CHUNK_OVERLAP

All settings are centralized in app/config.py and loaded via Settings class.

Telemetry & Logging

Structured JSON logging to ./logs/rag_YYYYMMDD.log tracks:

• Query Events: Query, device, provider, model, latency, tokens, citations, has_context
• Comparison Events: Query, device, providers, latencies, tokens
• Ingestion Events: Device, source_file, pages, chunks, duration
• Error Events: Operation, error, details

Access via global telemetry instance from app.telemetry.

File Organization

app/
├── api/
│   └── routes.py              # FastAPI endpoints, request/response models
├── llm_providers/
│   ├── base.py                # LLMProvider abstract base class
│   ├── claude.py              # Anthropic Claude implementation
│   ├── openai_provider.py     # OpenAI implementation
│   └── gemini.py              # Google Gemini implementation
├── services/
│   ├── ingestion.py           # PDF → chunks → embeddings → vector store
│   ├── retrieval.py           # Query → semantic search → re-ranking
│   └── rag.py                 # Retrieval + LLM generation orchestration
├── vector_store/
│   ├── base.py                # VectorStore abstract base class
│   └── local_store.py         # FAISS-based local implementation
├── ui/                        # Static HTML demo UI
├── config.py                  # Pydantic Settings for env-based config
├── telemetry.py               # Structured JSON logging
└── main.py                    # FastAPI app initialization

Adding New Components

Adding a New Vector Store

1. Create app/vector_store/your_store.py:

   from app.vector_store.base import VectorStore, VectorStoreConfig
   
   class YourVectorStore(VectorStore):
       async def add_documents(self, texts, embeddings, metadata):
           # Implementation
           pass
       
       async def search(self, query_embedding, top_k, filter_metadata):
           # Implementation
           pass
       
       # Implement other abstract methods
   

2. Update app/api/routes.py → get_vector_store() to instantiate your store based on config

3. Update .env: VECTOR_STORE_TYPE=your_store

Adding a New LLM Provider

1. Create app/llm_providers/your_provider.py:

   from app.llm_providers.base import LLMProvider, LLMResponse
   
   class YourProvider(LLMProvider):
       async def generate(self, prompt, system_prompt):
           # Call your LLM API
           return LLMResponse(
               text=response_text,
               provider="your_provider",
               model="model_name",
               tokens_used=token_count,
               finish_reason="stop"
           )
       
       def get_provider_name(self):
           return "your_provider"
   

2. Update app/api/routes.py → get_llm_provider() to handle your provider name

3. Add API key to .env: YOUR_PROVIDER_API_KEY=...

Important Implementation Details

Embedding Model

The system uses all-MiniLM-L6-v2 by default (384 dimensions). If changing models:

• Update EMBEDDING_MODEL in .env
• Update embedding_dimension in VectorStoreConfig (if dimensions differ)
• Re-ingest all documents (embeddings are not compatible across models)

Authentication

API endpoints (except health check) require X-API-Key header matching API_KEY from .env. Default for dev: dev-key-123.

Device Management

Devices are identified by name strings (e.g., "Bair Hugger Model 775"). Device filtering in queries:

• Uses metadata filtering in vector store search
• Exact match on device metadata field
• Device names are case-sensitive

Context Window

The system retrieves TOP_K_CHUNKS (default: 5) chunks. Each chunk is ~512 characters. This provides ~2,500 characters of context to the LLM, well within all modern LLM context limits.

Troubleshooting

No Results Returned

• Check if PDFs have been ingested: GET /api/stats (requires API key)
• Verify device name matches exactly (case-sensitive)
• Check ./logs/rag_YYYYMMDD.log for errors

Provider Errors

• Verify API keys in .env
• Check provider-specific rate limits
• Use /api/compare to test all providers (gracefully handles missing keys)

Vector Store Issues

• Check ./data/vector_store/ exists and has write permissions
• FAISS index is created on first ingestion
• To reset: delete ./data/vector_store/ and re-run ingestion

API Endpoint Summary

All endpoints except health check require X-API-Key header.

• GET /api/ - Health check (no auth required)
• GET /api/stats - Vector store statistics (total chunks, devices, embedding dimension)
• POST /api/query - Query with single provider (body: {query, device?, provider?})
• POST /api/compare - Compare all available providers (body: {query, device?})
• POST /api/ingest - Ingest PDF from data/ directory (body: {device_name, filename})
• POST /api/export - Export answer as JSON for ServiceNow/Jira integration

See http://localhost:8000/docs for interactive API documentation.