What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models

Analysis

Why This Paper Matters

This paper addresses a fundamental yet overlooked issue in AI compliance monitoring: whether detectors actually read the rules they are supposed to enforce. The authors demonstrate that current compliance detectors—both guard models and activation probes—exhibit 'rule blindness,' meaning their verdicts are unaffected by the content of the governing rule. This is a critical finding because regulatory compliance in deployed language models is increasingly used as a legal and audit control. If a detector's decision does not depend on the stated rule, it cannot provide meaningful compliance assurance, potentially leading to regulatory violations and safety failures.

The paper's significance is amplified by its rigorous methodology. By constructing a crossed-rule benchmark that controls for surface features, the authors rule out alternative explanations and show that rule blindness is not an artifact of prior benchmark designs. This work challenges the AI community to develop detectors that genuinely reason about rules, rather than relying on spurious correlations. The release of a counterfactual protocol and benchmark provides a concrete way for others to test rule blindness in their own systems, fostering a culture of accountability.

Technical Contributions

• Rule Blindness Identification: The paper systematically tests deletion, permutation, and substitution of the governing rule across multiple detectors, showing no change in accuracy. This is a robust demonstration of the failure.
• Crossed-Rule Benchmark: A purpose-built benchmark crossing two rules with two scenarios ensures that neither rule nor scenario alone predicts the label, isolating the effect of the rule. This design is a methodological advance over prior benchmarks.
• Internal Compliance Score (ICS): A training-free activation readout that requires only ten labeled pairs and a single projection. It is designed for scalable auditing without retraining, making it practical for large-scale evaluations.
• Counterfactual Protocol: A standardized protocol for testing rule blindness, enabling future researchers to audit their own detectors.
• Step-by-Step Reasoning: The paper shows that step-by-step reasoning escapes rule blindness, suggesting a path forward for building rule-aware detectors.

Results

The paper reports that all tested guards and activation probes, including a policy-conditioned guard that correctly cites the governing clause, show unchanged detection accuracy when the rule is deleted, permuted, or substituted. For example, swapping a rule for its permissive counterpart barely changes the verdict. The ICS, while inexpensive, fails to meet a pre-registered criterion for beating trivial baselines, and a bag-of-words model matches its pooled generalization exactly. However, ICS raises the mechanically verified pass rate when used to rank candidate responses, though an adaptive white-box attack removes this gain. These results underscore the difficulty of building rule-aware detectors and the need for more robust approaches.

Significance

The broader impact of this work is twofold. First, it exposes a critical vulnerability in current compliance monitoring systems, which could have legal and ethical implications for deployed AI. Second, it provides tools (benchmark and protocol) for the community to audit and improve detectors. The finding that step-by-step reasoning escapes rule blindness suggests that future detectors should incorporate explicit reasoning mechanisms. While ICS has limitations, its low cost makes it a viable tool for large-scale audits, potentially enabling broader oversight of AI systems. This paper is a call to action for the AI safety and compliance community to prioritize rule-awareness in detector design.