Think Deep, Not Just Long (2026)

Free

Longer CoT ≠ better reasoning — identifies "deep-thinking tokens" (high-revision tokens) as the true signal; enables cost-efficient test-time scaling

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About Think Deep, Not Just Long (2026)

This paper introduces a method to measure LLM reasoning effort by identifying 'deep-thinking tokens'—tokens where internal predictions undergo significant revisions in deeper model layers before convergence. The authors show that the proportion of such tokens (deep-thinking ratio) strongly and consistently correlates with accuracy across challenging mathematical and scientific benchmarks (AIME 24/25, HMMT 25, GPQA-diamond) and diverse models (GPT-OSS, DeepSeek-R1, Qwen3). This correlation is substantially stronger than length-based or confidence-based baselines. Based on this insight, the paper proposes Think@n, a test-time scaling strategy that prioritizes samples with high deep-thinking ratios, enabling early rejection of unpromising generations from short prefixes and reducing inference costs while maintaining or improving performance over standard self-consistency.

Key Features

Identifies deep-thinking tokens where internal predictions undergo significant revisions in deeper model layers

Introduces deep-thinking ratio as a robust proxy for reasoning quality, outperforming length- and confidence-based metrics

Proposes Think@n test-time scaling strategy that prioritizes high deep-thinking ratio samples

Enables early rejection of unpromising generations based on short prefixes, reducing inference costs

Validated on multiple challenging math and science benchmarks (AIME 24/25, HMMT 25, GPQA-diamond) and models

Pros & Cons

Pros

Deep-thinking ratio shows robust positive correlation with accuracy across benchmarks
Outperforms length-based and confidence-based methods for assessing reasoning quality
Think@n matches or exceeds self-consistency performance at lower cost
Allows early rejection of poor generations using only prefix tokens, saving compute

Cons

Requires access to internal model logits across layers to compute deep-thinking tokens, not available in all APIs
Method currently a research concept, not a readily packaged tool for end users

Best For

Efficient test-time compute scaling for LLMsImproving reasoning accuracy in mathematical problem-solvingReducing inference costs by filtering low-effort generations earlyScientific reasoning tasks requiring deep logical chains

FAQ

What are deep-thinking tokens?

Deep-thinking tokens are tokens where the model's internal predictions undergo significant revisions in deeper layers before converging on the final output. They indicate uncertainty or effort in the reasoning process.

What is Think@n?

Think@n is a test-time scaling strategy that generates multiple candidate responses, ranks them by deep-thinking ratio, and selects the final answer from the high-ratio subset, enabling early rejection of poor generations.

On which benchmarks was the method evaluated?

The method was evaluated on four challenging mathematical and scientific benchmarks: AIME 24, AIME 25, HMMT 25, and GPQA-diamond.

What models were used in the experiments?

The experiments used GPT-OSS, DeepSeek-R1, and Qwen3 reasoning-focused models.