Architecture Decision Record (ADR): RAG Security Scanner

Date: 2026-02-04 Status: Active Development Authors: Claude Code Team Repository: Ksushik/rag-security-scanner

Executive Summary

The RAG Security Scanner is an AI-powered document security analysis tool designed to detect PII, credentials, and malicious content in documents before they enter Retrieval-Augmented Generation (RAG) systems. This addresses the critical security gap in enterprise AI deployments and EU AI Act compliance requirements.

Business Context

Market Opportunity

• EU AI Act Compliance: New regulations requiring security audits of AI training data
• Enterprise AI Security: Growing need for document security in RAG systems
• Cost of Data Breaches: Average cost of AI-related data breach is $4.88M
• TAM: €2.1B European AI compliance market by 2027

Competitive Landscape Analysis

Solution	Focus	Differentiation
Pantheon-Security/medusa	Code security, 74 analyzers, general SAST	General code scanning
olegnazarov/rag-security-scanner	RAG vulnerability testing, prompt injection	Runtime testing, not document scanning
Our Solution	Document content security, EU AI Act compliance	Pre-ingestion document analysis

Key Differentiation:

• Document-level content analysis vs code-level security
• EU AI Act compliance focus vs general security
• Pre-ingestion scanning vs runtime vulnerability testing
• Enterprise compliance reporting vs developer tooling

Architecture Decisions

1. Technology Stack

Decision: Python-based microservices architecture

Rationale:

• Rich ecosystem for NLP/ML (spaCy, transformers, sklearn)
• Strong enterprise adoption for AI/ML workloads
• Excellent documentation and community support
• Easy integration with existing Python-based RAG systems

Components:

Core Engine (Python 3.11+)
├── Document Processors (PDF, DOCX, TXT, MD, HTML)
├── Security Analyzers (PII, Credentials, Malicious Content)
├── Compliance Engine (EU AI Act, GDPR)
└── Reporting Engine (JSON, PDF, HTML)

API Layer (FastAPI)
├── REST API for document submission
├── WebSocket for real-time scanning progress
└── Webhook support for CI/CD integration

Storage Layer
├── PostgreSQL (scan results, audit trails)
├── Redis (caching, job queues)
└── S3-compatible (document storage)

2. Document Processing Pipeline

Decision: Multi-stage async processing pipeline

Architecture:

Ingestion → Content Extraction → Security Analysis → Compliance Check → Reporting
     ↓              ↓                    ↓               ↓             ↓
File Upload → Text/OCR Extraction → Analyzer Execution → Rule Engine → Report Gen

Processing Stages:

1. Document Ingestion

   • Support: PDF, DOCX, TXT, MD, HTML, RTF, ODT
   • Size limits: 100MB per file, 1GB batch
   • Metadata extraction: Author, creation date, modification history

2. Content Extraction

   • Text extraction with position mapping
   • OCR for scanned documents (Tesseract)
   • Structured data extraction (tables, forms)
   • Image content analysis (embedded images)

3. Security Analysis

   • PII Detection (GDPR Article 4 compliance)
   • Credential Scanning (API keys, passwords, tokens)
   • Malicious Content Detection (injection patterns, suspicious URLs)
   • Language Model Safety (harmful content classification)

4. Compliance Engine

   • EU AI Act Article 10 requirements
   • GDPR data processing records
   • Industry standards (SOC2, ISO27001)
   • Custom compliance rules

3. Security Analyzer Architecture

Decision: Plugin-based analyzer system with ML models

Core Analyzers:

A. PII Detection Engine

• Technology: spaCy + custom NER models + regex patterns
• Coverage: GDPR Article 4 categories (names, addresses, IDs, biometric data)
• Languages: Multi-language support (EN, DE, FR, ES, IT)
• Confidence Scoring: ML-based confidence with explainability

B. Credential Scanner

• Technology: Regex patterns + entropy analysis + ML classification
• Coverage: API keys, passwords, tokens, certificates, connection strings
• Database: 10,000+ known credential patterns
• Context Analysis: Reduce false positives with semantic understanding

C. Malicious Content Detector

• Technology: Transformer-based classification + pattern matching
• Coverage: Injection attacks, phishing, malware indicators, suspicious URLs
• Real-time Updates: Threat intelligence feed integration
• Behavioral Analysis: Document modification pattern analysis

D. EU AI Act Compliance Checker

• Technology: Rule engine + policy templates
• Coverage: Article 10 (transparency), Article 13 (accuracy), Article 14 (oversight)
• Documentation: Automated compliance documentation generation
• Audit Trail: Complete scan history with version control

4. API Design

Decision: RESTful API with async processing

# OpenAPI 3.0 Schema
/api/v1/scan:
  post:
    summary: Submit documents for scanning
    requestBody:
      multipart/form-data:
        files: [file1, file2, ...]
        compliance_profile: "eu-ai-act" | "gdpr" | "custom"
        scan_profile: "fast" | "comprehensive" | "custom"
    responses:
      202: { scan_id, status_url, webhook_url }

/api/v1/scan/{scan_id}:
  get:
    summary: Get scan status and results
    responses:
      200: { status, progress, results, compliance_report }

/api/v1/scan/{scan_id}/report:
  get:
    summary: Download compliance report
    parameters:
      format: "json" | "pdf" | "html"

5. Deployment Architecture

Decision: Cloud-native with on-premises option

Production Deployment:

# Kubernetes Deployment
apiVersion: v1
kind: Deployment
metadata:
  name: rag-security-scanner
spec:
  containers:
  - name: api
    image: ksushik/rag-security-scanner:latest
    resources:
      requests: { memory: "2Gi", cpu: "1" }
      limits: { memory: "8Gi", cpu: "4" }
  - name: worker
    image: ksushik/rag-security-scanner-worker:latest
    resources:
      requests: { memory: "4Gi", cpu: "2" }

On-Premises Option:

• Docker Compose for single-node deployment
• Air-gapped deployment support
• Local model inference (no cloud dependencies)

Implementation Plan

Phase 1: MVP Core Engine (4 weeks)

Sprint 1-2: Foundation (2 weeks)

• Project setup and CI/CD pipeline
• Basic document processing (PDF, DOCX, TXT)
• Core PII detection with spaCy
• Simple REST API with FastAPI
• SQLite database for local testing
• Basic CLI interface

Sprint 3-4: Security Features (2 weeks)

• Credential scanning engine
• Basic malicious content detection
• Result aggregation and reporting
• Configuration management
• Unit test coverage >80%

Deliverables:

• Functional MVP scanning PDF/DOCX/TXT files
• CLI tool with basic commands
• REST API with async processing
• JSON report generation
• Docker containerization

Phase 2: Enterprise Features (6 weeks)

Sprint 5-6: Advanced Analysis (2 weeks)

• Multi-language PII detection
• Advanced credential patterns
• ML-based content classification
• OCR support for scanned documents
• Performance optimization

Sprint 7-8: Compliance Engine (2 weeks)

• EU AI Act rule implementation
• GDPR compliance checks
• Custom compliance profiles
• Audit trail functionality
• Compliance report templates

Sprint 9-10: Production Ready (2 weeks)

• PostgreSQL/Redis integration
• Kubernetes deployment manifests
• Monitoring and observability
• Load testing and optimization
• Security hardening

Deliverables:

• Production-ready scanner with all core features
• EU AI Act compliance reporting
• Kubernetes deployment
• Performance benchmarks
• Security documentation

Phase 3: Enterprise Platform (8 weeks)

Sprint 11-14: Platform Features (4 weeks)

• Web dashboard and UI
• User management and RBAC
• API authentication and rate limiting
• Webhook integrations
• Batch processing interface

Sprint 15-18: Advanced Features (4 weeks)

• Custom rule development
• Integration with popular RAG frameworks
• Enterprise SSO integration
• Advanced reporting and analytics
• White-label deployment options

Deliverables:

• Complete enterprise platform
• Web UI for non-technical users
• Extensive integration options
• Advanced analytics and reporting
• Go-to-market materials

Technology Decisions

Core Dependencies

# requirements.txt
fastapi==0.104.1           # API framework
uvicorn==0.24.0           # ASGI server
spacy==3.7.2              # NLP processing
transformers==4.35.0      # ML models
sqlalchemy==2.0.23        # Database ORM
redis==5.0.1              # Caching/queues
celery==5.3.4             # Task processing
pytest==7.4.3            # Testing framework
docker==6.1.3             # Container management
kubernetes==28.1.0        # K8s integration

Model Selection

• PII Detection: spaCy en_core_web_lg + custom trained NER
• Content Classification: DistilBERT fine-tuned on security datasets
• Language Detection: fastText language identification
• OCR: Tesseract with language packs

Infrastructure

• Container Registry: GitHub Container Registry
• CI/CD: GitHub Actions with automated testing
• Monitoring: Prometheus + Grafana
• Logging: Structured logging with ELK stack
• Security: Vulnerability scanning with Trivy

Risk Assessment

Technical Risks

Risk	Impact	Mitigation
False Positives	High	ML confidence scoring + human review workflow
Performance at Scale	Medium	Async processing + horizontal scaling
Model Accuracy	High	Continuous retraining + multiple validation datasets
Memory Usage	Medium	Streaming processing + efficient data structures

Business Risks

Risk	Impact	Mitigation
Regulatory Changes	Medium	Modular compliance engine + regular updates
Competition	Medium	Focus on enterprise features + superior UX
Market Adoption	High	Open source community + strong documentation
Technical Debt	Low	Clean architecture + comprehensive testing

Success Metrics

Technical KPIs

• Accuracy: >95% precision, >90% recall on PII detection
• Performance: <30s scan time for 100-page document
• Scalability: 1000+ concurrent scans
• Availability: 99.9% uptime

Business KPIs

• Adoption: 100+ GitHub stars in 3 months
• Enterprise: 5+ pilot customers in 6 months
• Compliance: 100% EU AI Act Article 10 coverage
• Community: 1000+ downloads in first month

Future Roadmap

Q2 2026: Advanced Features

• Real-time document monitoring
• Integration with Microsoft 365, Google Workspace
• Advanced ML models for context understanding
• Multi-tenant SaaS deployment

Q3 2026: Enterprise Platform

• Advanced analytics and dashboards
• Custom model training interface
• Marketplace for compliance templates
• Partner ecosystem development

Q4 2026: Global Expansion

• Multi-region compliance (US, APAC)
• Industry-specific templates (Healthcare, Finance)
• Advanced threat intelligence integration
• AI-powered compliance recommendations

Conclusion

This ADR establishes the foundation for building a market-leading RAG security scanner focused on EU AI Act compliance and enterprise document security. The architecture balances technical excellence with business viability, positioning us to capture significant market share in the emerging AI compliance sector.

The phased approach allows for rapid MVP deployment while building toward a comprehensive enterprise platform. Strong focus on accuracy, performance, and compliance ensures product-market fit for our target enterprise customers.

---

Next Steps:

1. Set up GitHub repository and CI/CD pipeline
2. Implement Phase 1 Sprint 1 deliverables
3. Establish testing and quality standards
4. Begin community building and documentation