When Is Benchmark Contamination Detectable? Information Limits and Power-Calibrated Audits

Analysis

Why This Paper Matters

Benchmark contamination—where test data leaks into training sets—undermines the validity of machine learning evaluations. Behavioral contamination detectors, which probe model outputs for signs of memorization or anomalous performance, are widely used to audit benchmarks. However, when such a detector reports "no evidence" of contamination, practitioners often interpret this as a clean benchmark. This paper argues that this inference is flawed: a negative result may simply reflect an underpowered audit. By formalizing the distinction between a clean benchmark and a low-power audit, the authors address a critical blind spot in evaluation methodology.

The significance is twofold. First, it provides a theoretical foundation for understanding the limits of contamination detection, drawing on information theory and statistical power analysis. Second, it offers practical guidance for designing audits that can meaningfully distinguish between these two scenarios. As benchmarks become larger and more influential, ensuring their integrity is paramount, and this work equips the community with tools to avoid false confidence.

Technical Contributions

• Formalization of the detection problem: The paper models contamination detection as a hypothesis testing problem, where the null hypothesis is a clean benchmark and the alternative is contamination. It defines the power of an audit as the probability of correctly detecting contamination when present.
• Information-theoretic limits: The authors derive bounds on the maximum achievable power given the properties of the benchmark, the contamination level, and the detector's capabilities. These limits show that some contamination is fundamentally undetectable with behavioral methods.
• Power-calibrated audits: They propose a procedure that estimates and reports the power of an audit alongside its verdict. This allows practitioners to interpret negative results appropriately—either as evidence of cleanliness (when power is high) or as inconclusive (when power is low).
• Practical implications: The framework suggests guidelines for designing audits with sufficient power, such as increasing the number of probes or using more sensitive detectors.

Results

The paper's main results are theoretical. It establishes that the detectability of contamination is governed by an information-theoretic threshold: if the contamination signal is too weak relative to the noise in detector outputs, no audit can reliably detect it. The authors show that power-calibrated audits can achieve near-optimal detection rates under certain conditions. They also demonstrate through simulations or case studies (though the abstract does not provide specific numbers) that ignoring power can lead to drastically overestimating benchmark cleanliness. For example, an audit with 20% power would fail to detect contamination 80% of the time, yet a negative result would be misinterpreted as evidence of cleanliness.

Significance

This work has broad implications for how the AI community evaluates and reports benchmark contamination. It calls for a shift from binary "contaminated/clean" verdicts to probabilistic statements that account for audit power. By highlighting the information limits of detection, it encourages more honest and rigorous evaluation practices. Ultimately, this can lead to more trustworthy benchmarks and a better understanding of model capabilities. The framework is likely to influence future benchmark design and auditing standards, making it a valuable contribution to the field of machine learning evaluation.