Semblance AI — Development Roadmap

See specs/Semblance-MVP-Plan-v2.md for full technical specification.

Iteration 0: Scaffolding

• Repo cleanup — remove legacy files, submodules
• Rust workspace structure with crate scaffolding
• CI/CD pipeline (fmt, clippy, test)
• OpenAPI stub, JSON schemas
• README, CLAUDE.md, PLAN.md
• Basic vault directory initializer (bridge to Iteration 1)

Iteration 1: Vault Storage, Spaces, Artifacts

• Vault initialization with encryption (Argon2id key derivation + XChaCha20-Poly1305)
• macOS Keychain integration for DEK storage
• SQLite schema v0 with WAL mode
• Spaces CRUD (hierarchical containers)
• Artifact ingest pipeline
• Content-addressed encrypted blob store
• Deduplication (same file → same blob)
• Tags model and artifact tagging

Exit criteria: Import 1,000 files without corruption. Dedup verified. Listing < 100ms for 10k artifacts.

Iteration 2: Extraction and Chunking

• PDF text extraction (text layer)
• Markdown and plaintext extraction
• Sentence-aware chunking (512 tokens, 64 overlap)
• Chunk byte offset tracking for citation
• Derived text stored as encrypted blob
• Re-extraction skipped for unchanged artifacts

Exit criteria: Queryable chunk corpus exists. Extraction is deterministic.

Iteration 3: Embeddings and Vector Index

• EmbeddingProvider trait + ONNX MiniLM-L6-v2 implementation
• VectorIndex trait + sqlite-vec implementation (vectors in same SQLite DB)
• Embedding pipeline (chunk → vector → index)
• Vector ID ↔ chunk ID mapping
• Model version tracking for future reindexing

Exit criteria: 10k+ chunks indexed. Query returns relevant chunks < 100ms. Recall@10 > 0.70 on golden set.

Iteration 4: Lens Policies and Policy-Filtered Retrieval

• LensPolicy model and CRUD
• Policy evaluation engine
• Policy-filtered retrieval via SQL WHERE clauses (sqlite-vec enables single-query filtering)
• Space isolation enforcement
• Tag include/exclude, content type, sensitivity ceiling filtering

Exit criteria: Blocked spaces never appear in results. Policy filtering degrades Recall@10 by < 10%.

Iteration 5: Disclosure, Citations, Receipts

• Disclosure modes: summary, excerpt, full
• Citation generation with byte offsets
• Receipt construction and storage (JSON)
• Receipt export (HTML)
• v0.0 CLI checkpoint — end-to-end validation in terminal

Exit criteria: Every query produces a receipt. Citations have valid byte offsets. Core value proposition demoed without UI.

Iteration 6: Gateway

• Axum localhost server bound to 127.0.0.1
• Agent registration and pairing (6-char code)
• Token signing (Keychain-stored key)
• Session management (create, expire, revoke)
• Query endpoint with policy enforcement
• SSE activity stream
• Rate limiting (100 queries/min/session)

Exit criteria: Mock agent can register, pair, query, and be revoked. Sessions expire at TTL.

Iteration 7: Mac App

• SwiftUI shell with vault picker and unlock
• Spaces sidebar (tree view) and artifact list
• Import via drag-drop and file dialog
• Lens builder wizard
• Agent grant dialog with approval flow
• Live session monitor with activity stream
• Receipt viewer with artifact links and export

Exit criteria: Full end-to-end user story works from import through receipt review.

Iteration 8: Hardening and Ship

• Transaction wrapping, WAL checkpoint management
• Crash recovery tests
• Batch embedding during ingest
• Background indexing queue with progress indicators
• Performance profiling against budgets
• Security audit (encryption, policy enforcement, token handling)
• macOS code signing and notarization
• DMG packaging with first-run experience
• semblance diag diagnostics command
• Schema versioning for vault migration

Exit criteria: Fresh install works on macOS 13+. All performance budgets met. No P0 bugs.

---

Testing Strategy

Every iteration includes tests at all relevant layers:

• Unit tests: Module-level in #[cfg(test)] mod tests
• Integration tests: Cross-crate in tests/integration/
• Property-based tests (proptest): Policy evaluation, encryption round-trips, chunking invariants
• Security tests: Blocked space isolation, token expiry, revocation, encryption without passphrase
• Retrieval quality tests: Golden set evaluation (Recall@5, Recall@10, MRR)
• Performance tests: Budget compliance for ingest, query, startup, memory

MVP Definition of Done

• Spaces and Lens policies exist and are easy to use
• Retrieval is semantic and policy-filtered
• Agents access only via session tokens with declared purpose and TTL
• Every query yields citations and a receipt
• User can revoke access immediately
• Everything works fully offline
• Recall@10 > 0.80 on golden evaluation set
• All performance budgets met
• Fresh install works on macOS 13+
• No P0 bugs remaining