LLMTrace Implementation TODO

# LLMTrace Implementation TODO

**Generated from:** `docs/FEATURE_ROADMAP.md`
**Updated:** 2026-02-15
**Methodology:** RALPH loops — each loop spawns a Claude Code agent with strict quality gates, reviewed by lead engineer before merge.

---

## Status Legend
- ⬜ Not started
- 🔄 In progress
- ✅ Done
- ❌ Blocked

---

## Acceptance Criteria (Literature-Anchored)
These criteria define when a task can be marked ✅. If any criterion is not met, status must remain 🔄 or ⬜.

Input Security (IS): Must implement the specific algorithmic behaviors described in the literature, not heuristic approximations. For IS-001–IS-003, MOF requires token-wise bias detection, debiasing data generation, and retraining with reported over-defense gains in `docs/research/injecguard-over-defense-mitigation.md`. For IS-006/IS-007, thresholds must be calibrated at 0.1/0.5/1% FPR with TPR reporting per `docs/research/security-state-of-art-2026.md`. For IS-010/IS-011, WordNet-style synonym expansion and true lemmatization are required, not regex-only stems, per `docs/research/dmpi-pmhfe-prompt-injection-detection.md`. For IS-024–IS-029, adversarial robustness must include attack-specific defenses and calibration beyond normalization per `docs/research/bypassing-llm-guardrails-evasion.md`.

DMPI-PMHFE Architecture (DMPI-001–DMPI-006): The fusion pipeline matches the paper's dual-channel design. All 6 deviations resolved: average pooling (DMPI-001), 2 FC layers (DMPI-002), 10 binary heuristic features with paper keyword sets (DMPI-003, DMPI-005), repetition threshold >=3 (DMPI-004), `is_*` naming convention (DMPI-006). See Loop 12a and `docs/research/dmpi-pmhfe-prompt-injection-detection.md` for full specification. ML-001 (fusion training) is no longer blocked by DMPI deviations.

Tool/Agent Security (AS): Tool boundary defenses must parse/sanitize with LLM-based extraction and CheckTool-style triggering detection, not heuristic filters, per `docs/research/defense-tool-result-parsing.md` and `docs/research/indirect-injection-firewalls.md`. Multi-agent defense requires an explicit coordinator + guard multi-pass architecture (and second opinion path) rather than a single-pass heuristic pipeline, per `docs/research/multi-agent-defense-pipeline.md`. Pattern enforcement must detect plan compliance and routing by trust level as defined in `docs/research/design-patterns-securing-agents.md`.

Output Security (OS): HaluGate-style token-level detection requires ModernBERT token classification and NLI explanation layer, not heuristic or sentence-only checks, per `docs/research/security-state-of-art-2026.md`. Streaming safety must use partial-sequence models and progressive confidence (SCM), not re-running full-text detectors, per `docs/research/security-state-of-art-2026.md`. CodeShield parity requires Semgrep integration and coverage beyond basic static rules, per `docs/research/security-state-of-art-2026.md`.

Privacy/Protocol/Multimodal (PR/AS/MM/SA): Membership inference, poisoning, MINJA, and protocol exploit defenses must match the threat models in `docs/research/prompt-injections-to-protocol-exploits.md`. Multimodal defenses must include OCR and modality-specific detectors as described in the same literature. Policy language and taint/blast-radius controls must align with `docs/research/llmtrace-defense-pipeline-design.md`.

Evaluation (EV): Benchmarks must implement the named suites with published dataset sizes and result formatting per `docs/research/benchmarks-and-tools-landscape.md` and `docs/research/wasp-web-agent-security-benchmark.md`.

Non-Functional Requirements (NFR): Security-critical detections must be deterministic and testable, with clear latency budgets where specified (e.g., HaluGate sentinel 12ms class, token-level detection 76–162ms) from `docs/research/security-state-of-art-2026.md`. Any ML integration must include reproducible model loading, configuration, and tests demonstrating expected metrics.

## Phase 1: Critical / Quick Wins

### Loop 1 — Unicode Evasion Defenses
> Close the 100% ASR emoji smuggling and upside-down text gaps

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-020 | Emoji normalisation/stripping — 100% ASR, zero current defense | Low | ✅ `a62855b` |
| IS-021 | Upside-down text mapping — 100% jailbreak evasion | Low | ✅ `a62855b` |
| IS-022 | Unicode tag character stripping (U+E0001–U+E007F) | Low | ✅ `a62855b` |
| IS-031 | Diacritics-based evasion defense — accent marks | Low | ✅ `a62855b` |
| IS-015 | Braille encoding evasion defense | Low | ✅ `a62855b` |

### Loop 2 — NotInject Benchmark + 3D Evaluation
> Establish over-defense baseline and evaluation framework (current dataset: 210 samples, difficulty split 90/60/60)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-004 | NotInject-style over-defense benchmark dataset (339 samples, 3 difficulty levels) | Low | ✅ |
| IS-005 | Three-dimensional evaluation metrics (benign/malicious/over-defense) | Low | ✅ `33b3f55` |
| EV-002 | NotInject evaluation runner (dataset complete: 339 samples) | Low | ✅ |
| EV-010 | Paper-table output format for results | Low | ✅ `33b3f55` |

### Loop 3 — FPR-Aware Threshold Optimisation
> Evaluate at deployment-realistic FPR operating points

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-006 | FPR-aware threshold optimisation — evaluate at 0.1%, 0.5%, 1% FPR | Medium | ✅ `fpr_monitor.rs` |
| IS-007 | Configurable operating points (high-precision / balanced / high-recall) | Low | ✅ (R8) |

### Loop 4 — Canary Token System
> Detect system prompt leakage in responses

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| SA-002 | Canary token injection and leakage detection | Low | ✅ `5b43d93` |

### Loop 5 — Tool Registry & Classification
> Foundation for agent security features

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-008 | Tool registry with security classification (category, risk score, permissions) | Medium | ✅ `eae4ca3` |
| AS-015 | Action-type rate limiting | Low | ✅ `eae4ca3` |

### Loop 6 — Context Window Flooding Detection
> DoS prevention (OWASP LLM10)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-017 | Context window flooding detection | Low | ✅ `9997962` |

---

### Loop R0 — Scaffold the Workspace
> Create workspace, crates, and baseline repo hygiene

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL0-01 | Initialize Cargo workspace and required crates | Medium | ✅ |
| RL0-02 | Add root README, .gitignore, rustfmt config | Low | ✅ |
| RL0-03 | Ensure crates compile cleanly | Medium | ✅ |

### Loop R1 — Core Types & Traits
> Define foundational core types and traits

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL1-01 | Core types: TraceEvent, TraceSpan, TenantId, SecurityFinding, SecuritySeverity, LLMProvider, ProxyConfig | Medium | ✅ |
| RL1-02 | Core traits: StorageBackend (or successors), SecurityAnalyzer | Medium | ✅ |
| RL1-03 | Error types via thiserror, serde on public types, timestamp types | Medium | ✅ |
| RL1-04 | Serialization roundtrip tests | Medium | ✅ |

### Loop R2 — SQLite Storage Backend
> Implement SQLite storage backend

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL2-01 | Sqlite storage implementation with migrations | Medium | ✅ |
| RL2-02 | store/query/health_check for traces | Medium | ✅ |
| RL2-03 | Integration tests with temp DB | Medium | ✅ |

### Loop R3 — Basic Prompt Injection Detection
> Regex-based prompt injection detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL3-01 | RegexSecurityAnalyzer request/response scanning | Medium | ✅ |
| RL3-02 | Patterns: system override, role injection, base64, PII | Medium | ✅ |
| RL3-03 | Comprehensive tests for known attacks | Medium | ✅ |

### Loop R4 — Transparent Proxy Core
> Core proxy flow and async analysis

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL4-01 | HTTP proxy flow (accept, parse, forward, return) | High | ✅ |
| RL4-02 | Support OpenAI-compatible routes | Medium | ✅ |
| RL4-03 | Async trace capture + security analysis | Medium | ✅ |
| RL4-04 | Circuit breaker and health endpoint | Medium | ✅ |
| RL4-05 | YAML config loading | Medium | ✅ |

### Loop R5 — Streaming SSE Support
> Stream passthrough and token tracking

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL5-01 | Detect streaming requests and forward SSE | High | ✅ |
| RL5-02 | Incremental token/TTFT tracking | High | ✅ |
| RL5-03 | Integration tests with mock SSE upstream | Medium | ✅ |

### Loop R5.5 — Storage Layer Refactor
> Repository pattern split for traces/metadata/cache

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL5-501 | Split storage traits into trace/metadata/cache | High | ✅ |
| RL5-502 | Add tenant/config/audit types | Medium | ✅ |
| RL5-503 | Storage composite + profile factory | Medium | ✅ |
| RL5-504 | SQLite repos for traces + metadata, in-memory cache | High | ✅ |
| RL5-505 | Proxy integration with new storage profile config | High | ✅ |

### Loop R6 — Configuration & CLI
> CLI and config validation

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL6-01 | Clap CLI with proxy/validate subcommands | Medium | ✅ |
| RL6-02 | Example config + env var overrides | Medium | ✅ |
| RL6-03 | Structured logging | Low | ✅ |

### Loop R7 — Python Bindings
> PyO3 bindings and tests

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL7-01 | PyO3 crate setup + Python API | High | ✅ |
| RL7-02 | Python tests via maturin | Medium | ✅ |

### Loop R8 — Integration Test & Polish
> End-to-end proxy + docs

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL8-01 | Integration test with proxy + mock upstream | High | ✅ |
| RL8-02 | Top-level README, LICENSE | Low | ✅ |

## Phase 2: Major Features

### Loop 7 — Tool-Boundary Firewalling
> The "minimize & sanitize" approach — reported low ASR on paper benchmarks (scope-specific)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-001 | Tool-Input Firewall (Minimizer) — heuristic minimizer; no LLM-based minimization | High | 🔄 |
| AS-002 | Tool-Output Firewall (Sanitizer) — heuristic sanitizer; no LLM-based parsing | High | 🔄 |
| AS-003 | Tool context awareness — tool context defined but not used in minimizer/sanitizer | Medium | 🔄 |
| AS-004 | ParseData — extract minimal required data from tool outputs (LLM-based parsing not implemented) | High | 🔄 |
| AS-005 | Format constraint validation — heuristic rules only (no schema-driven parsing) | Medium | 🔄 |
| AS-006 | CheckTool — detect tool-output-triggered tool calls (heuristic only) | High | 🔄 |
| AS-007 | Tool output sanitization against injection triggers (heuristic only) | High | 🔄 |

### Loop 8 — Model Ensemble Diversification
> Replace single-model reliance with multi-architecture ensemble

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| ML-002 | InjecGuard model integration | Medium | ✅ `10a2369` |
| ML-003 | Meta Prompt Guard 2 integration (86M + 22M) | Medium | ✅ `10a2369` |
| ML-006 | Multi-model ensemble voting with diverse architectures — InjecGuard wired as 3rd detector, majority voting replaces union merge | Medium | ✅ |
| ML-004 | PIGuard model integration | Medium | ✅ |
| ML-007 | Model hot-swapping without proxy restart | Medium | ⬜ |

### Loop 9 — Action-Selector Pattern Enforcement
> Provable security patterns at proxy level

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-010 | Action-Selector pattern — enforce action allowlists at proxy level | Medium | ✅ `89ba304` |
| AS-012 | Context-Minimization — strip unnecessary context | Medium | ✅ `89ba304` |
| AS-011 | Plan-then-execute pattern detection | High | ⬜ |
| AS-014 | Plan compliance monitoring for declared security patterns | High | ⬜ |
| AS-013 | Dual LLM routing for trusted/untrusted data | High | ⬜ |
| AS-016 | Trust-based routing by data source | High | ⬜ |

### Loop 10 — Multi-Agent Defense Coordination
> Coordinator + Guard architecture — reported low ASR on paper benchmarks (scope-specific)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-020 | Coordinator agent — pre-input classification (policy/heuristic pipeline only) | High | 🔄 |
| AS-021 | Guard agent — post-generation validation (policy/heuristic pipeline only) | High | 🔄 |
| AS-022 | Hierarchical coordinator pipeline (safe routing/refusal) | High | ⬜ |
| AS-023 | Second opinion pass for borderline cases (no true multi-agent LLM pass) | Medium | 🔄 |
| AS-024 | Policy store — centralised security rules (in-memory, not externalized) | Medium | 🔄 |
| AS-025 | Multi-step action correlation across requests | High | ✅ |
| AS-026 | Multi-turn persistence detection for gradual bypass attempts | High | ✅ |

### Loop 11 — MCP Protocol Monitoring
> First-mover in protocol-level security

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-030 | MCP monitoring — detect manipulation and server-side attacks | High | ✅ `mcp_monitor.rs` |
| AS-035 | Toxic Agent Flow defense — GitHub MCP vulnerability (generic MCP scanning only) | Medium | 🔄 |
| AS-036 | ToolHijacker defense — tool selection manipulation (generic MCP scanning only) | High | 🔄 |

### Loop 12 — Advanced Prompt Injection Detection
> Synonym expansion, lemmatisation, P2SQL

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-010 | Synonym expansion for attack patterns (manual synonym regex, not WordNet) | Medium | 🔄 |
| IS-011 | Lemmatisation before pattern matching (basic stemming, not true lemmatization) | Low | 🔄 |
| IS-012 | P2SQL injection detection (regex only, no structured SQL parsing) | Medium | 🔄 |
| IS-013 | Long-context jailbreak detection (position-aware sliding window) | High | ⬜ |
| IS-014 | Automated jailbreak defense (GPTFuzz-style genetic templates) | High | ⬜ |
| IS-016 | Multi-turn extraction detection (session-aware probing) | High | 🔄 |
| IS-040 | Data format coverage expansion (17 formats) | Medium | ⬜ |
| IS-041 | Multi-language trigger detection | High | ⬜ |
| IS-018 | "Important Messages" header attack hardening | Low | 🔄 |
| IS-050 | Perplexity-based anomaly detection for GCG-optimized strings in tool outputs | Medium | ⬜ |
| IS-051 | Adaptive monitoring scope (input-only vs hybrid) to control attack surface | Medium | ⬜ |
| IS-052 | Adversarial string propagation blocking in tool outputs (perplexity threshold) | High | ⬜ |

### Loop 12a — DMPI-PMHFE Architecture Alignment
> Resolve 6 architectural deviations between codebase and DMPI-PMHFE paper (arXiv 2506.06384). All 6 resolved (DMPI-001, DMPI-002, DMPI-003, DMPI-004, DMPI-005, DMPI-006).
> Loop 15 (Fusion Training Pipeline) is no longer blocked by DMPI deviations.
> Reference: `docs/research/dmpi-pmhfe-prompt-injection-detection.md`

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| DMPI-001 | **Average pooling instead of CLS token** — Implemented `masked_mean_pool()` in `ml_detector.rs`. Added `PoolingStrategy` enum (Cls/MeanPool), defaulting to MeanPool. BERT and DeBERTa paths both use attention-mask-aware average pooling over all non-padding tokens, matching paper spec. `DebertaV2ContextPooler` is now optional (only loaded for Cls strategy). Architecture doc: `docs/architecture/DMPI_001_AVERAGE_POOLING.md`. | Medium | :white_check_mark: |
| DMPI-002 | **2 FC layers instead of 3** — Removed `HIDDEN_2` and `fc3`; collapsed to `fc1(783->256)->ReLU->fc2(256->2)->SoftMax` matching paper spec. Input dim changes from 783 to 778 once DMPI-003 is also applied (768 + 10 = 778). Architecture doc: `docs/architecture/DMPI_002_TWO_FC_LAYERS.md`. | Medium | :white_check_mark: |
| DMPI-003 | **10 binary features instead of 15 mixed** — Replaced 15-dim vector (8 binary + 7 numeric) with 10 binary features matching paper Appendix A. Removed all numeric features. Added keyword-based detection for `is_ignore`, `is_format_manipulation`, `is_immoral`. Reordered to paper spec. Architecture doc: `docs/architecture/DMPI_003_TEN_BINARY_FEATURES.md`. | High | :white_check_mark: |
| DMPI-004 | **Repetition threshold >=3 instead of >10** — Named constant `REPETITION_THRESHOLD = 3`. Word-level and phrase-level conditions changed to `>= REPETITION_THRESHOLD`. Expanded `COMMON_WORDS` (+37 words) and added `COMMON_PHRASES` exclusion list (29 common English bigrams) to control false positives at the lower threshold. | Low | :white_check_mark: |
| DMPI-005 | **Missing paper features: is_immoral, is_ignore, is_format_manipulation** — All 3 missing features now implemented as keyword-in-text checks in `feature_extraction.rs`. `is_ignore` (index 0): ignore, reveal, disregard, forget, overlook, regardless. `is_format_manipulation` (index 4): encode, disguising, morse, binary, hexadecimal. `is_immoral` (index 7): hitting, amoral, immoral, deceit, irresponsible, offensive, violent, unethical, smack, fake, illegal, biased. Resolved as part of DMPI-003. Architecture doc: `docs/architecture/DMPI_003_TEN_BINARY_FEATURES.md`. | Medium | :white_check_mark: |
| DMPI-006 | **Feature naming alignment to paper convention** — All 8 finding types renamed to paper's `is_*` convention: `flattery_attack->is_incentive`, `urgency_attack->is_urgent`, `roleplay_attack->is_hypothetical`, `impersonation_attack->is_systemic`, `covert_attack->is_covert`, `excuse_attack->is_immoral`, `many_shot_attack->is_shot_attack`, `repetition_attack->is_repeated_token`. Updated in `lib.rs`, `feature_extraction.rs`, and documentation. | Low | ✅ |

### Loop 13 — Hallucination Detection Upgrade
> HaluGate-style token-level detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| OS-001 | Token-level hallucination detection (ModernBERT) | High | ⬜ |
| OS-002 | NLI explanation layer for flagged spans | High | ⬜ |
| OS-003 | ModernBERT sentinel pre-classifier | Medium | ⬜ |
| OS-004 | Tool-call result as ground truth for fact-checking | Medium | ⬜ |
| OS-005 | Semantic entropy-based detection | High | ⬜ |
| OS-006 | Citation validation | High | ⬜ |
| ML-005 | ModernBERT support (for token/sentinel classifiers) | High | ⬜ |

### Loop 14 — Content Safety Expansion
> Llama Guard integration, bias detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| OS-022 | Llama Guard 3 integration (14 harm categories) | Medium | ⬜ |
| OS-021 | Bias detection in responses | Medium | ⬜ |
| OS-020 | Constitutional classifiers for output moderation | High | ⬜ |
| OS-023 | Language detection for unexpected output switches | Low | ⬜ |
| OS-024 | Sentiment analysis for manipulative content | Low | ⬜ |
| OS-030 | CodeShield-style code security expansion | High | 🔄 |
| OS-031 | Semgrep rule integration for code outputs | High | ⬜ |
| OS-032 | Supply chain security in code (typosquatting, confusion) | High | ⬜ |

### Loop 15 — Fusion Training Pipeline
> Train the fusion classifier with real data

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| ML-001 | Joint end-to-end training for fusion FC layer | High | ✅ |
| ML-014 | Curated training dataset (61k benign + 16k injection) | Medium | ✅ |
| IS-001 | Token-wise bias detection for over-defense | High | ⬜ |
| IS-002 | Adaptive debiasing data generation (1–3 token combos) | High | ⬜ |
| IS-003 | MOF retraining pipeline on debiased data | High | ⬜ |
| ML-010 | MOF training pipeline (token bias → debiasing → retraining) | High | ⬜ |
| ML-011 | Data-centric augmentation across 17 formats | Medium | ⬜ |
| ML-015 | GradSafe integration | High | ⬜ |
| ML-016 | GCG adversarial sample generation (Python/PyTorch tooling; shared with EV-017) | High | ⬜ |
| ML-020 | ONNX runtime support for inference | Medium | ⬜ |
| ML-021 | INT8/INT4 quantized model loading | Medium | ⬜ |
| ML-022 | Batched inference for GPU utilization | Medium | ⬜ |

### Loop 16 — Benchmark Evaluation Suite
> Evaluate against all major benchmarks

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| EV-001 | AgentDojo evaluation (97 environments) | Medium | ⬜ (requires Python framework, not static dataset) |
| EV-003 | InjecAgent evaluation (2108 indirect injection samples) | Medium | ✅ |
| EV-004 | ASB evaluation (400 agent security attack samples) | Medium | ✅ |
| EV-005 | WASP evaluation | Medium | ⬜ (requires live web environment) |
| EV-006 | CyberSecEval 2 prompt injection evaluation (251 attack samples per DMPI-PMHFE [28]) | Medium | ✅ `7ce0cf9` |
| EV-007 | MLCommons AILuminate jailbreak benchmark (1200 demo prompts) | Medium | ✅ |
| EV-008 | HPI attack approximation (55 instances, 8-category taxonomy from arXiv:2509.14285) | Low | ✅ (best-effort 55-attack approximation) |
| EV-009 | Automated CI-integrated benchmark runner | Medium | ✅ `b15f4f0` |
| EV-011 | safeguard-v2 evaluation (2060 samples) | Low | ✅ |
| EV-012 | deepset-v2 evaluation (355 samples) | Low | ✅ |
| EV-013 | Ivanleomk-v2 evaluation (610 samples) | Low | ✅ |
| EV-014 | BIPIA evaluation (400 samples: 200 benign + 200 indirect injection, 3 scenarios) | Medium | ✅ |
| EV-015 | HarmBench evaluation (400 harmful behaviors, jailbreak/safety ASR) | Medium | ✅ |
| EV-016 | AgentDojo Slack suite adaptive attack evaluation (Agent-as-a-Proxy resilience, 89 samples) | High | ⬜ |
| EV-017 | Multi-objective GCG adversarial robustness red-team testing against LLMTrace ensemble | High | ⬜ |
| EV-018 | Cross-model transfer attack resistance testing across ensemble members | Medium | ✅ |
| EV-019 | Tensor Trust prompt hijacking/extraction evaluation (1000 sampled attacks) | Low | ✅ |
| EV-020 | Harelix mixed-techniques evaluation (1174 samples, tri-class) | Low | ❌ (dataset deleted from HuggingFace) |
| EV-021 | Jackhhao jailbreak-classification over-defense test (1306 samples, balanced) | Low | ✅ |

---

### Loop R9 — REST Query API

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL9-01 | Trace/span query endpoints + pagination | High | ✅ |
| RL9-02 | Security findings endpoint | Medium | ✅ |
| RL9-03 | API tests | Medium | ✅ |

### Loop R10 — LLM Provider Auto-Detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL10-01 | Provider detection by path/header/host | Medium | ✅ |
| RL10-02 | Provider-specific response parsing | Medium | ✅ |
| RL10-03 | Provider detection tests | Medium | ✅ |

### Loop R11 — Cost Estimation Engine

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL11-01 | Pricing table + estimate_cost API | Medium | ✅ |
| RL11-02 | Custom pricing config | Medium | ✅ |
| RL11-03 | Tests for pricing | Medium | ✅ |

### Loop R12 — Alert Engine (Webhooks)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL12-01 | Webhook alerting with thresholds + cooldown | Medium | ✅ |
| RL12-02 | Mock webhook tests | Medium | ✅ |

### Loop R13 — Tenant Management API

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL13-01 | Tenant CRUD endpoints + audit | High | ✅ |
| RL13-02 | Auto-create tenant on first request | Medium | ✅ |
| RL13-03 | API tests | Medium | ✅ |


### Loop R14 — ClickHouse TraceRepository

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL14-01 | ClickHouse TraceRepository implementation | High | ✅ |
| RL14-02 | Feature-gated ClickHouse tests | High | ✅ |

### Loop R15 — PostgreSQL MetadataRepository

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL15-01 | Postgres MetadataRepository + migrations | High | ✅ |
| RL15-02 | Postgres integration tests | High | ✅ |

### Loop R16 — Redis CacheLayer

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL16-01 | Redis CacheLayer implementation | Medium | ✅ |
| RL16-02 | Cache TTL and invalidation tests | Medium | ✅ |

### Loop R17 — Data Retention & Purging

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL17-01 | Retention policies + purge job | Medium | 🔄 |
| RL17-02 | Purge audit logging | Medium | ⬜ |

### Loop R18 — Agent Action Analysis

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL18-01 | AgentAction model + auto-parse tool calls | High | ✅ |
| RL18-02 | Actions reporting API + query filters | High | ✅ |
| RL18-03 | Action security analysis + storage | High | ✅ |
| RL18-04 | Python SDK action reporting | Medium | ✅ |

## Phase 3: Research Frontier

### Loop 17 — Multimodal Security
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| MM-001 | Image injection detection | High | ⬜ |
| MM-004 | OCR-based text extraction from images | Medium | ⬜ |
| MM-002 | Audio injection detection | High | ⬜ |
| MM-003 | Cross-modal consistency checking | High | ⬜ |
| MM-005 | Steganography detection (image/audio) | High | ⬜ |
| MM-006 | Video frame injection detection | High | ⬜ |

### Loop 18 — Protocol Security (A2A/ANP)
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| AS-031 | A2A protocol security | High | ⬜ |
| AS-032 | ANP protocol security | High | ⬜ |
| AS-033 | Dynamic trust management | High | ⬜ |
| AS-034 | Inter-agent trust verification | High | ⬜ |

### Loop 19 — Streaming Content Monitor
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| OS-010 | Purpose-built partial-sequence detection models | High | ⬜ |
| OS-011 | Training-inference gap mitigation (partial sequence training) | High | ⬜ |
| OS-012 | Token-level harm annotations | High | ⬜ |
| OS-013 | Progressive confidence scoring | Medium | ⬜ |

### Loop 20 — Advanced Privacy
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| PR-001 | Membership inference defense | High | ⬜ |
| PR-002 | Data extraction prevention | High | ⬜ |
| PR-003 | Federated learning poisoning defense | High | ⬜ |
| PR-004 | Vector/embedding poisoning detection | High | ⬜ |
| PR-005 | RAG retrieval anomaly monitoring | Medium | ⬜ |
| PR-006 | Multi-language PII detection (non-Latin scripts) | High | 🔄 |
| PR-007 | Context-aware PII enhancement (lemma-based boosting) | Medium | 🔄 |
| PR-009 | Compliance mapping to GDPR/HIPAA/CCPA entities | Medium | 🔄 |
| PR-010 | Memory poisoning detection (MINJA) | High | ⬜ |
| PR-011 | Cross-session state integrity | High | ⬜ |
| PR-008 | Custom PII entity type plugins | Medium | ⬜ |
| PR-012 | Speculative side-channel defense | High | ⬜ |

### Loop 21 — Policy Language
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| SA-001 | Declarative policy specification (Colang/OPA-style) | High | ⬜ |
| SA-003 | Taint tracking | High | ⬜ |
| SA-004 | Blast radius reduction for tool access | Medium | ⬜ |
| SA-005 | Backdoor detection (prompt/parameter level) | High | ⬜ |
| SA-006 | Composite backdoor detection (CBA-style) | High | ⬜ |
| SA-007 | Data poisoning detection (PoisonedRAG) | High | ⬜ |
| SA-008 | Social engineering simulation defense | High | ⬜ |
| SA-009 | Contagious recursive blocking defense | High | ⬜ |
| SA-010 | GuardReasoner integration | High | ⬜ |

### Loop 22 — Adversarial ML Robustness
| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| IS-024 | AML evasion resistance (TextFooler, BERT-Attack, BAE) — normalization only, no attack-specific defenses | High | 🔄 |
| IS-025 | Ensemble diversification against transferability — no transferability testing or training | High | 🔄 |
| IS-026 | Adversarial training integration (TextAttack samples) | High | ⬜ |
| IS-027 | Adaptive thresholding for evasion indicators | Medium | ⬜ |
| IS-028 | Multi-pass normalisation (aggressive + conservative + semantic-preserving) | Medium | 🔄 |
| ML-012 | Adversarial training on TextAttack samples | High | ⬜ (needs training pipeline) |
| ML-013 | Robust training with Unicode/character injection samples | High | ⬜ |
| IS-029 | Confidence calibration (Platt scaling) — temperature scaling only | Medium | 🔄 |
| IS-023 | Character smuggling variants (comprehensive unicode exploitation) | Medium | 🔄 |
| IS-030 | Word-importance transferability mitigation | High | ⬜ |

### Loop 23 — E2E Accuracy Optimization (Post Stress Test)
> After wiring OperatingPoint, threshold filtering, over-defence suppression, and score capping for single-detector findings, the E2E stress test reached **83.7% accuracy, 84.7% F1** on a 153-sample corpus (79 malicious, 74 benign) from 13+ benchmark datasets. The remaining 15 FPs and 10 FNs require ML-level fixes documented below.
> Reference: `docs/FEATURE_ROADMAP.md` section 3.4.4 for full analysis.
>
> **Review findings (2026-02-15, AI Engineer + MLOps Engineer):**
> - Combined ML-030 + ML-033 impact is NOT additive; realistic combined: -7 to -12 FPs.
> - ML-030 must precede ML-033 (calibrating before fine-tuning is wasted work).
> - ML-030 triggers ML-001 re-evaluation (fusion classifier needs re-validation after base model changes).
> - ML-033 supersedes IS-029 (Loop 22). IS-029 remains for temperature scaling only; ML-033 adds proper Platt scaling.
> - IS-060 elevated to P0 (4 FNs, largest single FN category, indirect injection is most dangerous for agent systems).
> - IS-070 elevated to P1 (shell injection in agent contexts is high-severity).
> - ML-034 elevated to P1 (encoding bypass is an active evasion vector).
> - Acceptance criterion for ALL items: full benchmark suite recall must not decrease by >1pp.
> - MLOps prerequisites (OPS-001 through OPS-008) must be addressed before deploying model changes.
> - Recommended execution order: ML-032 + ML-034 (patch evasion vectors) -> ML-033 (calibrate existing system) -> IS-070 (expand detection) -> ML-030 (model fine-tuning, highest risk last).

**Infrastructure already wired (this session):**
- `SecurityAnalysisConfig.operating_point` + `SecurityAnalysisConfig.over_defence` config fields
- `EnsembleSecurityAnalyzer::filter_by_thresholds()` applying per-category confidence gates
- `EnsembleSecurityAnalyzer::apply_over_defence()` suppressing auxiliary-only findings (no injection corroboration)
- Single-detector score cap at 60 (Medium) in `add_security_finding()`
- 3 new regex patterns: `roleplay_lets` (jailbreak), `authority_claim_update` (is_systemic), `disable_safety` (prompt_injection)

**Dependency chain:**
```
OPS-001..OPS-008 (prerequisites)
    |
    v
ML-032 + ML-034 (patch evasion vectors, low risk)
    |
    v
ML-033 (calibrate existing system, supersedes IS-029)
    |
    v
IS-060 + IS-070 (new detection capabilities)
    |
    v
ML-030 (fine-tune DeBERTa, highest risk)
    |
    v
ML-001 re-evaluation (fusion classifier re-validation)
    |
    v
ML-031 (multilingual calibration, depends on language detection infra)
```

**MLOps prerequisites (must complete before deploying ML changes):**

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| OPS-001 | **Externalize SecurityConfig to file/env** — All thresholds, operating points, and model identifiers loadable from config file or environment variables. Hardcoded values serve only as fallback defaults. Required before ML-033 (recalibrated thresholds currently require code change and rebuild). | Medium | ⬜ |
| OPS-002 | **Pin model revisions with SHA** — Use `hf_hub` revision/commit SHA parameters for all model downloads. Add SHA256 integrity verification of SafeTensors files. Current code downloads latest revision on cold start, risking silent behavior changes. | Low | ⬜ |
| OPS-003 | **Model inference metrics** — Expose per-model inference latency histograms, raw score distributions, classification outcome counters, ensemble agreement rate. Essential for validating model changes in production. | Medium | ⬜ |
| OPS-004 | **Model version manifest** — `models.toml` declaring model name, revision SHA, expected SafeTensors SHA256, deployment timestamp. Enables rollback by reverting manifest to previous version. | Low | ⬜ |
| OPS-005 | **CI regression test gate** — CI step that loads ensemble, runs fixed regression set (50-100 canonical examples), asserts no accuracy regression beyond threshold. Blocks merge on failure. | Medium | ⬜ |
| OPS-006 | **Training infrastructure for ML-030** — Define training environment (Python/PyTorch), GPU provisioning, SafeTensors export validation step ensuring layer-name compatibility with Candle loader. Define artifact registry for fine-tuned weights. | High | ⬜ |
| OPS-007 | **Expand calibration dataset to 1,000+ samples** — 153 stress test samples is insufficient for Platt scaling. Collect stratified samples across injection types. Separate calibration holdout from ML-030 training set (at least 30% of NotInject reserved for calibration). | Medium | ⬜ |
| OPS-008 | **Shadow-mode inference** — Run new model ensemble in parallel without affecting response path, log predictions for offline comparison. Required for safe validation of ML-030 fine-tuned model before production cutover. | High | ⬜ |

**ML accuracy work items:**

| ID | Feature | Complexity | Priority | Status |
|----|---------|-----------|----------|--------|
| ML-032 | **Short-input confidence scaling** — For inputs < 10 tokens, scale confidence threshold linearly from 0.95 (at 1 token) to normal threshold (at 10 tokens). Do NOT bypass ML entirely to avoid blind spots for short attacks like "Ignore all previous instructions" (5 tokens). Estimated impact: -1 FP. | Low | P1 | ⬜ |
| ML-034 | **Encoding decoder preprocessor** — Before ML inference, apply decoding pipeline: base64, rot13, leetspeak, hex, binary, upside-down text, Cyrillic homoglyphs. Add content-type heuristic before decoding (skip base64 if string contains spaces/punctuation). Specify latency cap (5ms max). Must define integration plan with existing `jailbreak_detector.rs` encoding detection (augment, not replace). 7/11 encoding evasion test cases detected (64%); 4 misses are encoded payloads without plaintext injection markers. | Medium | P1 | ⬜ |
| ML-033 | **Confidence recalibration (Platt scaling)** — Apply Platt scaling (logistic regression) to recalibrate DeBERTa output probabilities. Supersedes IS-029 temperature scaling. Requires OPS-007 (1,000+ calibration samples). Calibration set MUST be disjoint from ML-030 training set. Specify per-model vs post-ensemble calibration. Re-derive operating point thresholds after calibration (current HighRecall/Balanced/HighPrecision values become invalid). Estimated impact: -2 to -4 FPs. Depends on: OPS-007. | Medium | P1 | ⬜ |
| IS-060 | **Spotlighting/datamarking for indirect injection** — Split input into instruction zones and data zones using configurable boundary markers. Apply injection detection only to data zones. Sub-tasks: (a) zone boundary detection heuristics for common data formats (HTML tables, email headers, CSV, JSON data fields), (b) config-declared boundary support, (c) ensemble integration (feed datamarking results into existing voting). Targets 4 BIPIA FNs (40% of all FNs). Reference: `docs/research/spotlighting-indirect-injection-defense.md` (datamarking reduces ASR from >50% to <3%). | High | P0 | ⬜ |
| IS-070 | **Shell command injection detection** — Detect dangerous shell commands (curl with exfiltration, python -c with socket, wget, reverse shell, rm -rf) in prompt content. Extend existing RL3-02 regex patterns (do not duplicate). Distinct from prompt injection; targets 2 FN code execution attacks. Critical for agent systems with tool-use capabilities. | Medium | P1 | ⬜ |
| ML-030 | **DeBERTa fine-tuning on NotInject dataset** — Fine-tune `protectai/deberta-v3-base-prompt-injection-v2` using 339 NotInject samples + 15 stress test FPs + 10-20 "creative writing instruction" samples as hard negatives. Mix with full training set (61k benign + 16k injection from ML-014) to prevent catastrophic forgetting. Training: 3 epochs, lr=2e-5, batch_size=16. Reserve 20% of NotInject for validation. Acceptance criteria: F1 >= 0.88 on held-out set, no per-class recall regression > 2%, full benchmark suite pass. Estimated impact: -5 to -10 FPs. Depends on: OPS-002, OPS-004, OPS-005, OPS-006. Triggers: ML-001 re-evaluation. | High | P0 | ⬜ |
| ML-031 | **Multilingual calibration** — Two sub-tasks: (a) add language detection to ensemble pipeline (e.g., `lingua-rs` or trigram detector), (b) calibrate per-language confidence thresholds using holdout set. Collect 1,000+ benign Chinese samples (traditional + simplified, technical/conversational/educational). Fine-tuning is a separate future item. Estimated impact: -2 FPs. Depends on: ML-030. | Medium | P2 | ⬜ |

---

### Loop R19 — ML Prompt Injection Detection (Candle)

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL19-01 | Candle ML detector + ensemble integration | High | ✅ |
| RL19-02 | ML config wiring + fallback | Medium | ✅ |
| RL19-03 | Benchmark + tests | Medium | 🔄 |

### Loop R20 — OpenTelemetry Ingestion Gateway

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL20-01 | OTLP/HTTP endpoint + mapping | High | ✅ |
| RL20-02 | OTEL ingestion tests | Medium | ✅ |

### Loop R21 — Web Dashboard

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL21-01 | Next.js dashboard scaffolding + pages | High | ✅ |
| RL21-02 | API client + charts + Docker | High | ✅ |

### Loop R22 — CI/CD Pipeline

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL22-01 | CI workflow (fmt/clippy/test) | Medium | ✅ |
| RL22-02 | Release workflow + image scan | Medium | ✅ |

### Loop R23 — RBAC & Auth

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL23-01 | API keys + role enforcement | High | ✅ |
| RL23-02 | Tenant isolation | High | ✅ |

### Loop R24 — Compliance Reporting

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL24-01 | Report generator + API | High | ✅ |
| RL24-02 | Optional PDF export | Medium | ⬜ |

### Loop R25 — gRPC Ingestion Gateway

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL25-01 | gRPC ingestion server + proto | High | ✅ |
| RL25-02 | Streaming ingestion support | High | ✅ |

### Loop R26 — Kubernetes Operator + Helm

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL26-01 | Helm chart + deployment docs | High | ✅ |
| RL26-02 | Optional CRD operator | High | ⬜ |

### Loop R27 — WASM Bindings

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL27-01 | wasm-bindgen crate + JS API | Medium | ✅ |
| RL27-02 | WASM tests | Medium | ✅ |

### Loop R28 — Node.js Bindings

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL28-01 | napi-rs bindings + TS types | Medium | ✅ |
| RL28-02 | Node tests | Medium | ✅ |


### Loop R29 — Statistical Anomaly Detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL29-01 | Anomaly detector + config | High | ✅ |
| RL29-02 | Alert integration + tests | High | ✅ |

### Loop R30 — Real-time Streaming Security Analysis

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL30-01 | Streaming incremental analysis | High | ✅ |
| RL30-02 | Mid-stream alerting tests | High | ✅ |

### Loop R31 — Expanded PII Detection

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL31-01 | International PII patterns + suppression | High | ✅ |
| RL31-02 | PII redaction modes + tests | High | ✅ |

### Loop R32 — ML PII via NER

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL32-01 | NER model integration + ensemble | High | ✅ |
| RL32-02 | NER tests | Medium | ✅ |

### Loop R33 — ML Inference Monitoring + Warm-up

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL33-01 | Inference timing + preload | Medium | ✅ |
| RL33-02 | Warm-up tests | Medium | ✅ |

### Loop R34 — Multi-Channel Alerting

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL34-01 | Slack (Block Kit) + PagerDuty (Events API v2) done; Email channel TODO | High | 🔄 |
| RL34-02 | Deduplication done; escalation stub only (no full escalation policy engine) | High | 🔄 |

### Loop R35 — Externalize Pricing + OWASP Tests

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL35-01 | Pricing config externalization | Medium | ✅ |
| RL35-02 | OWASP LLM Top 10 test suite | High | ✅ |


### Loop R36 — Graceful Shutdown + Signal Handling

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL36-01 | SIGTERM/SIGINT handling + task drain | High | ✅ |
| RL36-02 | Shutdown tests | Medium | ✅ |

### Loop R37 — Prometheus Metrics Endpoint

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL37-01 | Metrics endpoint + instrumentation | High | ✅ |
| RL37-02 | Metrics tests | Medium | ✅ |

### Loop R38 — Database Migration Management

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL38-01 | Migration tooling + CLI | High | ✅ |
| RL38-02 | Migration tests | Medium | ✅ |

### Loop R39 — Secrets Hardening + Startup Probe

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL39-01 | Secrets hardening + startup probe | Medium | ✅ |

### Loop R40 — Integration Tests in CI + Container Scanning

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL40-01 | Compose-based integration tests in CI | High | ✅ |
| RL40-02 | Container scanning in release | Medium | ✅ |

### Loop R41 — Per-tenant Rate Limiting + Compliance Persistence

| ID | Feature | Complexity | Status |
|----|---------|-----------|--------|
| RL41-01 | Tenant rate limiting middleware | High | ✅ |
| RL41-02 | Compliance report persistence + API | High | ✅ |
## Quality Gates (enforced on every loop)

1. **cargo fmt --all --check** — zero diffs
2. **cargo clippy --workspace -- -D warnings** — zero warnings
3. **cargo test --workspace** — zero failures (pre-existing failures must be fixed)
4. **Lead engineer review** — diff reviewed before commit
5. **CI green** — verified after push

## Notes

- IS-007 (Configurable operating points) completed in R8 commit `41e219b`. Fully wired to proxy config and ensemble in 2026-02-15 session: `SecurityAnalysisConfig.operating_point` field drives `EnsembleSecurityAnalyzer::with_operating_point()`, `filter_by_thresholds()` applies per-category confidence gates, `over_defence` flag enables auxiliary-only suppression.
- R11 (code_security module) completed in commit `b08dccc`, tests fixed in `aa9ab98`
- Each loop targets a coherent feature set that can be tested independently
- Phase 1 focuses on closing critical 100% ASR gaps and establishing evaluation baseline
- RALPH quality policy: no placeholders/mocks; if spec requires ML, implement real ML inference (regex fallback only when model weights unavailable).
- AS-004/AS-006/AS-007 are 🔄 because literature expects LLM-based parsing/sanitization for tool outputs; current implementation is heuristic only.
- AS-020/AS-021/AS-023/AS-024 are 🔄 because literature expects multi-agent LLM coordination; current implementation is heuristic/policy-only.
- IS-024/IS-027/IS-028/IS-029 are 🔄 because only normalization/temperature scaling exists (no attack-specific defenses or Platt scaling). ML-033 (Loop 23) supersedes IS-029 for Platt scaling; IS-029 remains for temperature-scaling-only scope.
- PR-006 is 🔄 because full non-Latin PII coverage and a custom-entity plugin architecture are not fully implemented.
- Tool parsing expectations come from `docs/research/defense-tool-result-parsing.md` and `docs/research/indirect-injection-firewalls.md`.
- Multi-agent expectations come from `docs/research/multi-agent-defense-pipeline.md`.
- Adversarial robustness expectations come from `docs/research/bypassing-llm-guardrails-evasion.md`.
- Over-defense mitigation expectations come from `docs/research/injecguard-over-defense-mitigation.md`.
- Benchmark coverage expectations come from `docs/research/benchmarks-and-tools-landscape.md` and `docs/research/wasp-web-agent-security-benchmark.md`.
- CyberSecEval 2 benchmark expectations (EV-006) come from `docs/research/cyberseceval2-llm-security-benchmark.md`. The 251 attack sample count is sourced from DMPI-PMHFE (arXiv 2506.06384) which used the CyberSecEval 2 prompt injection dataset; the full paper covers additional suites (500 code interpreter abuse prompts, exploit generation, FRR).
- BIPIA benchmark expectations (EV-014) come from `docs/research/bipia-indirect-prompt-injection-benchmark.md`. First indirect prompt injection benchmark (KDD 2025, arXiv 2312.14197): 86,250 test prompts, 50 attack types, 25-model baseline. Boundary token defense (`<data>`/`</data>`) is most impactful intervention (1064% ASR increase without it) and is implementable at proxy level (relevant to AS-001/AS-002).
- Agent-as-a-Proxy attack implications (EV-016) come from `docs/research/agent-as-a-proxy-attacks.md`. Monitoring-based defenses (including LLMTrace proxy monitoring) are fundamentally fragile: 90%+ ASR via GCG-optimized adversarial strings. Validates that structural defenses (AS-001/AS-002 sanitization, boundary tokens) are more robust than observation-based monitoring. High-perplexity detection in tool outputs is a viable countermeasure.
- IS-050 -> IS-052 dependency: IS-052 (adversarial string propagation blocking) depends on IS-050 (perplexity-based anomaly detection) for surprisal scoring. IS-050 must be implemented first. IS-052 runs before AS-002 in the tool-output sanitization pipeline.
- IS-050 -> IS-051 implicit dependency: IS-051 (adaptive monitoring scope) auto-switches to input-only mode when IS-050 detects sustained high-perplexity anomalies in tool outputs (suggests active adaptive attack). IS-050 must be implemented first for auto-switching; manual override works independently.
- ML-016 and EV-017 share GCG Python/PyTorch offline tooling (`tools/gcg/` or `scripts/adversarial/`). Not part of the Rust proxy runtime.
- EV-016 and EV-001 share AgentDojo benchmark infrastructure. EV-016 focuses on Slack suite (89 samples) with adaptive (GCG) attacks; EV-001 covers the full 97 environments.
- EV-018 depends on ML-006 (ensemble must be wired before transfer resistance can be tested).
- ML-016 (GCG adversarial sample generation) is in Loop 15 (Fusion Training Pipeline). Requires Python/PyTorch offline tooling, not Rust proxy code. Shared with EV-017.
- Token-level perplexity detection expectations (IS-050) come from `docs/research/token-level-perplexity-detection.md`. PGM-based per-token detection with GPT-2 124M (CPU-only, <1GB) achieves perfect sequence-level detection and 0.93+ token-level F1. O(n) DP algorithm. Core implementation reference for IS-050.
- Perplexity-based attack detection expectations (IS-050) come from `docs/research/perplexity-based-attack-detection.md`. Two-feature LightGBM (PPL + token length) achieves 99.1% F2 on GCG attacks. GCG mean PPL 3525 vs benign ~30-45. Perplexity alone is insufficient (false positives on code/non-English); token length as second feature resolves this.
- Task Shield alignment expectations (ML-016) come from `docs/research/task-shield-alignment-defense.md`. Task-alignment defense ("does this serve the user?") achieves 2.07% ASR with 69.79% utility on GPT-4o. ContributesTo scoring at message boundaries. Directly informs ML-016 goal-drift detector design; provides EV-016 baseline comparison targets.
- Spotlighting expectations (IS-004, AS-001/AS-002) come from `docs/research/spotlighting-indirect-injection-defense.md`. Datamarking reduces ASR from >50% to <3% with zero NLP quality impact. Dynamic/randomized tokens essential. Encoding (base64) achieves 0% ASR but requires GPT-4-class models. Validates and extends boundary tag approach.
- Instruction hierarchy expectations (IS-004, SA-003) come from `docs/research/instruction-hierarchy-defense.md`. Privilege hierarchy (system > user > tool) via SFT+RLHF. +63.1 pp on system message extraction defense. Validates proxy-level boundary tags as complement to model-level hierarchy. Over-refusal is main trade-off (-22.7 pp).
- DMPI-001–DMPI-006 (Loop 12a) were prerequisites for ML-001 (Loop 15). All 6 deviations are now resolved; the fusion classifier architecture matches the DMPI-PMHFE specification. See `docs/research/dmpi-pmhfe-prompt-injection-detection.md` for the authoritative paper breakdown.
- DMPI-003 and DMPI-005 resolved together: feature vector is now 10 binary dimensions matching paper Appendix A. See `docs/architecture/DMPI_003_TEN_BINARY_FEATURES.md`.
- DMPI-006 (naming) resolved: all 8 finding types renamed to paper's `is_*` convention.
- EV-002 is ✅ because the NotInject dataset is 339 samples with equal difficulty tiers (113/113/113).