Preprint
Machine Learning

AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses

Cheng Qian, Wenting Zhao, Liangwei Yang, Heng Wang, Jielin Qiu, Heng Ji, Silvio Savarese, Huan Wang, Shelby Heinecke
August 12, 2026
arXivPDF

0

Citations

0

Influential Citations

—

Venue

2026

Year

Abstract

Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.

Analysis

Why This Paper Matters

This paper challenges the conventional assumption that capability transfer from large to small models must occur during training. By demonstrating that a stronger model can build inference-time harnesses that significantly boost a weaker model's performance without any parameter updates, it opens a new paradigm for model efficiency and deployment. This is particularly relevant in scenarios where retraining is impractical due to cost or data constraints.

The focus on Theory-of-Mind benchmarks is a clever choice, as these tasks require complex reasoning and social inference, making them a strong testbed for capability transfer. The near-doubling of performance (0.49 to 0.91) is striking and suggests that many reasoning failures in smaller models are not due to missing knowledge but rather to unstable reasoning and output formatting, which can be corrected via scaffolding.

Technical Contributions

  • Test-time scaffolding framework: Introduces a clear two-stage process where a builder model iteratively refines a harness using a small validation set, then applies it to the test set.
  • Mechanistic analysis: Provides a detailed decomposition of why the harness works, showing that gains come from offloading reasoning to deterministic code, benchmark-specific routing, and strict answer-format enforcement, not from encouraging more reasoning or sampling.
  • Scaling insights: Demonstrates that builder-model reasoning effort (e.g., number of refinement rounds) monotonically improves harness quality, and that platform effects are modest relative to builder capability.
  • Heterogeneous gains: Shows that weaker target models benefit the most, suggesting that harnesses can help close the capability gap.

Results

The paper reports a substantial average improvement from 0.49 to 0.91 across four Theory-of-Mind benchmarks, representing an 85% relative increase. This is achieved with only 5% of the data used for validation during harness construction. The analysis reveals that the primary drivers are deterministic code offloading and answer-format enforcement, while increased reasoning or sampling has minimal impact. Additionally, the study finds that builder-model reasoning effort improves harness quality monotonically, and that platform effects are modest, indicating that the builder's capability is the key factor.

Significance

This work has significant implications for AI deployment, offering a low-cost alternative to distillation that can be applied on-the-fly. It also provides a new lens on model capabilities, suggesting that many errors are due to reasoning instability rather than knowledge gaps. Future work could extend this approach to other domains and explore automated harness discovery, potentially leading to more efficient and adaptable AI systems.

test-time-scaffoldingcapability-transfertheory-of-mindinference-time-harnessdistillationllm-evaluationprompt-engineering
ReasoningBenchmarks

Related Papers