Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection

Analysis

Why This Paper Matters

Benchmark contamination is a growing concern in machine learning, as language models are increasingly trained on massive web-scale datasets that may inadvertently include evaluation benchmarks. Contaminated benchmarks can inflate performance metrics and mislead researchers about true model capabilities. Existing detection methods—n-gram overlap, likelihood-based membership inference, and canary strings—each have significant practical limitations. N-gram overlap can miss semantically equivalent but lexically different contamination. Likelihood-based membership inference requires reference models or training data that are often unavailable. Canary strings must be inserted before training, which is impossible for retrospective audits of already-trained models.

This paper introduces a new approach called excess separability, which leverages the internal representations of language models to detect contamination. By probing the residual stream—the sequence of hidden states across transformer layers—the method aims to identify whether a model has memorized benchmark examples. The key insight is that contaminated examples may be more separable in the model's representation space than clean examples, and this excess separability can be isolated from nuisance factors such as example difficulty or domain.

The work is timely and relevant for AI practitioners who rely on benchmark results to compare models. If the method proves effective, it could become a standard tool for auditing model evaluations without requiring access to proprietary training data or pre-planned canary strings.

Technical Contributions

• Excess separability metric: A new measure that quantifies how much more separable contaminated examples are compared to clean ones in the model's residual stream.
• Nuisance-controlled probing: A technique that controls for confounding factors (e.g., example difficulty, domain, length) when probing residual-stream representations, ensuring that the detected signal is attributable to contamination rather than other variables.
• Contamination detection framework: An end-to-end pipeline for detecting benchmark contamination using only model internals and benchmark examples, without needing training data or canary strings.
• Evaluation on language model benchmarks: The method is demonstrated on standard benchmarks, though specific results are not detailed in the abstract.

Results

The abstract does not provide concrete metrics or comparisons. The paper likely presents experiments showing that excess separability can distinguish contaminated from clean examples with high accuracy, and compares favorably to existing methods. However, without access to the full paper, specific numbers (e.g., AUC, detection rates) cannot be reported. The authors may also show that the method is robust to various nuisance factors and works across different model sizes and architectures.

Significance

If the proposed method is validated, it could have a substantial impact on how the AI community ensures benchmark integrity. It offers a retrospective, training-data-free approach to contamination detection, which is particularly valuable for auditing third-party models. This could lead to more trustworthy evaluations and fairer comparisons between models. Additionally, the technique of nuisance-controlled residual-stream probing may inspire further work on interpreting and auditing model internals for other purposes, such as detecting data leakage or memorization. However, the method's reliance on model internals means it is only applicable to models where such access is available, which may limit its use for closed-source models. Overall, this paper contributes a novel angle to the important problem of benchmark contamination.