Demo Walkthrough

Quick Start (< 2 minutes)

Qdrant is pre-populated with 635 chunks from the 146-page Boeing 737 Operations Manual. Skip straight to querying.

1. Start services

cp .env.example .env
# Edit .env: add OPENROUTER_API_KEY, QDRANT_URL, QDRANT_API_KEY

docker compose up -d

2. Verify health

curl http://localhost:8000/health

Expected (after ~60-90s for model loading):

{
  "status": "healthy",
  "qdrant_connected": true,
  "collection_stats": {"points_count": 635, "status": "green"},
  "models_loaded": {"text_embedder": true, "vision_embedder": true, "llm_client": true}
}

3. Ask a question

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "What is the after start procedure for the isolation valve?"}'

---

Example Queries

Procedural question (text retrieval)

curl -s -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "We are doing a Flaps 15 takeoff. What is the first flap selection during retraction, and at what speed?"}'

For a Flaps 15 takeoff, the first flap selection during retraction is to Flaps 5, which is made at V2 + 15 knots [NP.20.33 — Takeoff Flap Retraction Speed Schedule].

Table lookup (structured data extraction)

curl -s -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "We are planning a Flaps 40 landing on a wet runway at a 1,000-foot pressure altitude airport. If the wind-corrected field length is 1,600 meters, what is our field limit weight?"}'

At 1,000 feet pressure altitude, 1,600 meters wind-corrected field length, and a wet runway, the landing field limit weight is 47.5 (1000 KG) [PD.12.1 — Landing].

Diagram question (vision retrieval)

curl -s -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "For a standard visual pattern, what three actions must be completed prior to turning base?"}'

Prior to turning base in a standard visual pattern, the following three actions must be completed: Gear Down, Flaps 15, and the Landing checklist. [NP.30.6 — Visual Traffic Pattern]

Systems knowledge (text + vision)

curl -s -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "I see an amber STAIRS OPER light illuminated on the forward attendant panel; what does that light indicate?"}'

The amber "STAIRS OPER" light illuminated on the forward attendant panel indicates that the airstair is in transit [1.30.19 — Controls and Indicators].

Cross-section question (multi-page retrieval)

curl -s -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "During the Descent and Approach procedure, what action is taken with the AUTO BRAKE select switch, and what is the Pilot Flying final action regarding the autobrake system during the Landing Roll procedure?"}'

During the Descent and Approach procedure, the AUTO BRAKE select switch is set to the desired brake setting [NP.20.35 — Amplified Procedures]. The Pilot Flying's final action regarding the autobrake system during the Landing Roll procedure is to disarm the autobrake prior to taxi speed [NP.20.39 — Landing Roll Procedure].

---

API Reference

POST /query

{
  "question": "string",
  "top_k": 10,
  "text_weight": 0.5,
  "vision_weight": 0.5
}

Returns: answer, citations (page_id + section_title), source chunks with scores, model used, token usage.

GET /health

Returns: Qdrant connection status, collection stats, loaded models, device info.

---

Running Evaluation

# Full eval (RAGAS + custom metrics)
python -m eval.run_evaluation

# Custom metrics only (no LLM cost, ~14s)
python -m eval.run_evaluation --custom-only

Results saved to eval/results/. See eval/results/latest_evaluation.json for the most recent run.

---

Ingestion (Not Required for Demo)

Qdrant ships pre-populated. To re-ingest from scratch:

EMBEDDING_DEVICE=auto python -m src.ingestion.ingest_orchestrator \
  --pdf "data/raw/Boeing B737 Manual.pdf"

Takes ~10 min (CUDA) or ~45 min (CPU). The pipeline supports --resume for checkpoint recovery if interrupted.