A security research paper published this week demonstrates a method to extract hidden reasoning traces from frontier AI models through API vulnerabilities, a disclosure that has rippled through the AI community with implications for privacy, security, and model distillation. The attack works against the APIs of every frontier AI company tested, according to the paper, and it exploits the very cryptographic signatures that labs have used to obscure their models' internal chain-of-thought since the launch of OpenAI's o1.
The findings, posted on Twitter on August 11, 2026, by researcher Alexander Panfilov (@kotekjedi_ml), show that encrypted reasoning blocks can be decoded and ported to different models, sessions, and users. The original disclosure tweet drew 1.52 million views, 226 replies, 1.17 thousand reposts, and 8.65 thousand likes within a day. A follow-up tweet detailing privacy findings added 610 thousand views, 129 replies, 350 reposts, and 4.96 thousand likes.
The Attack: Replaying Signed Reasoning Blocks
The technique is surprisingly straightforward once the encryption is bypassed. An attacker obtains a legitimate encrypted or signed reasoning block from a frontier model's API. That block is then replayed into a different request, often targeting a weaker model. The attacker uses an assistant prefill to transcribe the hidden content, samples repeatedly, discards refusals, and reconciles the transcriptions across attempts.
For Claude, the attack involves replaying a signed thinking block to Haiku 4.5 with an assistant prefill of '<thinking-copy>'. For GPT, the attacker injects an encrypted_content reasoning item multiple times into a fabricated conversation, samples up to 50 outputs, and bypasses the roughly 50-token verbatim-output threshold using chunked continuations. For Gemini, the attack attaches a thought_signature to a model turn with a '<thought>' prefill, followed by repeated sampling and reconciliation.
The vulnerabilities were responsibly disclosed, and several have already been fixed. However, the paper warns that the attack may still be possible despite those fixes. Matthew Green, a cryptographer, reported the first compromise in May, breaking down the attack and demonstrating a replay and side-channel via latency. That early report set the stage for the broader disclosure this week.
Can Bölük (@_can1357), another researcher, noted an alternative method to extract chain-of-thought via a 'deep_think' tool. By disabling thinking and providing that tool, Bölük found, a model can be induced to output its internal chain-of-thought format directly.
The attack's mechanics reveal a deeper structural issue. The cryptographic signatures that labs added to reasoning blocks were designed to prevent tampering and to signal authenticity, not to provide confidentiality. The paper argues that these signatures create a false sense of security. An attacker does not need to break the cryptography itself. Instead, the attacker treats the signed block as a black box, replays it into a context where the model is willing to transcribe it, and lets the model do the decryption work.
This replay approach works because the models themselves can read the reasoning blocks. The encryption is opaque to external observers but transparent to the model that generated it. By feeding the block back into a model with the right prefill instructions, the attacker converts the model into a decryption oracle. The repeated sampling and reconciliation steps are necessary because the model may refuse or produce partial transcriptions. The attacker discards refusals and merges overlapping transcriptions until the full reasoning trace emerges.
The paper's authors note that the attack's success rate varies by model family. Claude's signed thinking blocks are replayed to Haiku 4.5, a smaller and more compliant model. GPT's encrypted_content items are injected multiple times into a fabricated conversation, with chunked continuations used to bypass the verbatim-output threshold. Gemini's thought_signature is attached to a model turn with a '<thought>' prefill. Each approach is tailored to the specific API structure of the target lab.
The disclosure has also revived interest in side-channel attacks on reasoning traces. Matthew Green's May report demonstrated that latency itself can leak information about the reasoning process. A model that spends more time thinking produces a longer reasoning trace, and that trace length is observable through timing. Even if the content remains encrypted, the mere fact that a model thought for a certain duration reveals something about the complexity of the task. The paper suggests that such side-channels may be harder to fix than the direct replay attack.
Privacy Risks: Secrets Hidden in Reasoning Blocks
The privacy implications are severe. A scan of roughly 7,000 public traces found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data. Of those, 64 of the sensitive items appeared exclusively inside reasoning blocks, meaning they were never visible in the final output. Anyone who shared a trace publicly could have leaked credentials without knowing it.
Jonas Geiping (@jonasgeiping), a researcher who provided context on privacy risks and operational security, mentioned a leaked Hugging Face production key found in the scan. That key, like the others, was sitting inside a reasoning block that the model had never intended to reveal.
The finding underscores a fundamental problem: hidden chain-of-thought is not a reliable monitoring interface. If the reasoning traces contain secrets, and those traces can be extracted, then the encryption that labs have relied on is not a hard confidentiality barrier. One observer, @vipulved, framed the encryption of reasoning traces as a stateless distributed-inference protocol optimization rather than a security measure. That framing, if accurate, means the protection was never designed to withstand a determined adversary.
The paper's authors argue that labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces. The attack is a serious privacy and safety problem, though the analysis suggests it is not a scalable distillation path. Extracting a single reasoning trace requires significant effort, and the technique does not easily generalize to wholesale model copying.
Still, the disclosure has renewed interest in the interpretability Venn diagram of alignment, security, and chain-of-thought monitoring. If hidden reasoning can be extracted, then monitoring it becomes both more possible and more urgent.
The privacy scan itself raises questions about how traces are shared. Public traces are often posted by developers who want to demonstrate a model's behavior or debug a workflow. These traces frequently include the full conversation, including the reasoning blocks. The scan found that 64 sensitive items appeared exclusively inside reasoning blocks, meaning the model had considered those secrets during its internal deliberation but never surfaced them in the final answer. A developer sharing such a trace would have no way to know that the reasoning block contained a password or an API key.
The leaked Hugging Face production key is a concrete example. Hugging Face is a widely used platform for hosting models and datasets, and a production key would grant access to private repositories or billing accounts. The key was sitting inside a reasoning block, invisible to anyone who only looked at the final output. The scan's discovery of this key suggests that the problem is not hypothetical. Real credentials are flowing through reasoning blocks, and the extraction attack makes them accessible.
The paper also notes that the attack's privacy implications extend beyond credentials. Reasoning blocks can contain personal information about users, internal business data, or other sensitive content that the model considered during its deliberation. If an attacker can extract these blocks, they can harvest information that the model was never supposed to reveal. The paper argues that this makes the encryption of reasoning blocks a privacy issue, not just a security issue.
The response from the AI community has been mixed. Some researchers argue that the attack is a natural consequence of treating reasoning blocks as opaque containers. Others point out that the attack's complexity limits its practical impact. The paper's authors acknowledge both perspectives, noting that the attack is not a scalable distillation path but that it does lower the barrier for extracting specific reasoning traces. That is enough to worry model providers.
New Open-Weight Models: NVIDIA, Meta, Qwen, and Ling
Amid the security news, NVIDIA released Nemotron 3.5 Lightning, a 30B Mixture-of-Experts model with roughly 3 billion active parameters. The model offers up to 4x throughput and a 1 million token context window. It has 31.6B total parameters with 3.6B active, released under the OpenMDW-1.1 license with both NVFP4 and BF16 weights.
In pre-release endpoint testing, the model achieved a median serving speed of approximately 670 tokens per second. Its Intelligence Index score of 24 lands roughly in line with gpt-oss-120b. On GDPval-AA v2, it scored an Elo of 824, a major jump over Nemotron 3 Nano. On Terminal-Bench v2.1, it scored 24%, also a major improvement over its predecessor.
Artificial Analysis, the independent benchmarking organization, provided third-party analysis of the model. The model is available on Together AI, Ollama, Baseten, vLLM, and the Perplexity API, making it easy to deploy.
Harvey, the legal AI company, post-trained Lightning for legal work. On the Legal Agent Bench, the post-trained model improved from 0% to 8.3% on held-out tasks, beating Opus 4.6 and Nemotron 3 Ultra. Average output tokens were cut from 90,000 to 37,000, a dramatic efficiency gain for real-world legal workflows.
The release reinforces the open-model trend toward smaller, faster models tuned for high-volume tool use. NVIDIA positions Lightning as a "local agent workforce" that complements larger planning models. That positioning suggests the company sees the 30B parameter tier as the sweet spot for agentic workloads, where latency and cost matter more than raw benchmark scores.
The Harvey post-training is particularly notable because it demonstrates the value of domain-specific adaptation. Harvey took a general-purpose model and tuned it for legal work, achieving a jump from 0% to 8.3% on held-out Legal Agent Bench tasks. That improvement came alongside a massive reduction in output tokens, from 90,000 to 37,000. For legal workflows, where long outputs are common and costly, this efficiency gain is significant.
The model's serving speed of approximately 670 tokens per second in pre-release endpoint testing positions it as a high-throughput option. The 1 million token context window allows it to handle long documents, which is essential for legal and enterprise use cases. The mixture-of-experts architecture keeps active parameters low, which reduces inference cost while maintaining quality.
NVIDIA's positioning of Lightning as a "local agent workforce" is a deliberate contrast to the cloud-based frontier models. The company seems to be targeting developers who want to run agentic workloads on their own infrastructure, either for cost reasons or for data privacy reasons. The availability on Together AI, Ollama, Baseten, vLLM, and the Perplexity API makes it easy to deploy in a variety of environments.
The GDPval-AA v2 Elo of 824 and the Terminal-Bench v2.1 score of 24% both represent major improvements over Nemotron 3 Nano. These gains suggest that NVIDIA is iterating quickly on its smaller models, closing the gap with larger models on agentic benchmarks. The Intelligence Index score of 24, roughly in line with gpt-oss-120b, indicates that the model is competitive with other open-weight options in its size class.
Meta released Muse Glimmer 30B, an Apache 2.0-licensed dense multimodal model. It handles interleaved text and images, supports more than 100 languages, offers controllable reasoning effort, and was trained with an agent-focused approach.
The model quantizes well. At roughly 4-bit quantization, the language model drops below 20GB, fitting comfortably in 24-32GB of memory. On a single RTX 3090 with 24GB, it runs with a 262,144 token context, using about 22-23GB of VRAM. Generation speed ranges from 64 to 124 tokens per second under DFlash, the model's speculative decoding drafter. Prompt processing hits approximately 1,400 tokens per second. The model passed a 150,000 token needle test.
The KV cache is surprisingly compact at roughly 1.8 GiB for 131,000 tokens in F16, despite sliding window attention on all layers. That efficiency makes long-context work feasible on consumer hardware.
One observer called Muse Glimmer 30B "the strongest agentic model for its size for like three days before they release Qwen," a hedge that captures the pace of the mid-size model race. The quote, attributed to a researcher in the coverage, reflects the crowded field: Qwen, Google, NVIDIA, and Meta are all shipping models in this tier within days of each other.
Alexandr Wang, CEO of Scale AI, stated on X that an open-weight Muse Spark 1.2 release is coming soon. That would extend Meta's open-weight lineup further.
Some users report quality issues with Muse Glimmer 30B versus Laguna-S-2.1 in vLLM, suggesting the model's quantization behavior may vary by serving stack. The model may be "the strongest agentic model for its size for like three days before they release Qwen," but that window is closing fast.
The Muse Glimmer 30B release is notable for its hardware accessibility. Running a 30B multimodal model on a single RTX 3090 with a 262,144 token context is a significant achievement. The model uses about 22-23GB of VRAM, leaving room for the operating system and other processes. The DFlash speculative decoding drafter delivers generation speeds of 64 to 124 tokens per second, which is usable for interactive applications.
The compact KV cache is another engineering highlight. At roughly 1.8 GiB for 131,000 tokens in F16, the cache is small enough to allow long-context inference without excessive memory overhead. The sliding window attention on all layers contributes to this efficiency, though it may have implications for how the model handles information across long distances.
The 150,000 token needle test is a standard evaluation for long-context retrieval. Passing it means the model can find a specific piece of information buried in a long document, which is essential for agentic workloads that involve searching through large corpora. The model's support for more than 100 languages and interleaved text and images makes it a versatile option for multimodal tasks.
The crowded mid-size tier is a defining feature of this week's releases. Qwen, Google, NVIDIA, and Meta are all shipping models in the 30B range within days of each other. This competition is pushing down costs and improving quality, but it is also creating a confusing landscape for buyers. The observer's hedge that Muse Glimmer 30B would be "the strongest agentic model for its size for like three days before they release Qwen" captures the absurd pace of the race.
Alexandr Wang's statement about an open-weight Muse Spark 1.2 release adds another layer to Meta's strategy. Scale AI is a major player in the AI data and evaluation space, and Wang's endorsement suggests that the Muse family is gaining traction. The upcoming release would extend Meta's open-weight lineup further, giving developers more options in the mid-size tier.
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
The quality issues reported by some users in vLLM are a reminder that quantization behavior can vary by serving stack. A model that performs well in one environment may degrade in another. This variability complicates the task of comparing models and choosing a deployment target.
Qwen3.8-27B open weights are landing this week, according to the coverage. A ModelScope listing for Qwen3.8-2.4T-A95B shows a countdown of roughly 1 day and 9 hours, indicating a massive MoE model is also on the way.
inclusionAI released Ling-3.0-tiny, an 8B MoE model with about 1.3B active parameters. In FP8, it achieves 100-105 tokens per second on a DGX Spark and 86-90 tokens per second on an M4 Pro MacBook. Peak memory is about 8.34 GiB at 8K context, with support for 256K context. Its AA Bench score is 25.
On instruction-following benchmarks, Ling-3.0-tiny scores 63.61 on IFBench, 83.15 on Multi-IF, and 62.72 on BFCL-v4. Those numbers beat LFM2.5-8B-A1B, which scores 56.47, 79.93, and 49.73 respectively. The model is clearly competitive in its size class.
The Qwen3.8-27B open weights release is part of a broader pattern of rapid iteration from Alibaba's Qwen team. The ModelScope listing for Qwen3.8-2.4T-A95B, with a countdown of roughly 1 day and 9 hours, suggests that an even larger mixture-of-experts model is imminent. The 2.4T total parameter count with 95B active parameters would place this model in the frontier tier, competing with the largest open-weight models available.
Ling-3.0-tiny from inclusionAI is a different kind of release. At 8B total parameters with about 1.3B active, it is designed for efficiency. The FP8 performance numbers are impressive: 100-105 tokens per second on a DGX Spark and 86-90 tokens per second on an M4 Pro MacBook. Peak memory of about 8.34 GiB at 8K context means the model can run on modest hardware, and the 256K context support extends its utility for long-document tasks.
The instruction-following benchmark scores for Ling-3.0-tiny are competitive with LFM2.5-8B-A1B across the board. On IFBench, Ling scores 63.61 versus 56.47. On Multi-IF, it scores 83.15 versus 79.93. On BFCL-v4, it scores 62.72 versus 49.73. These numbers suggest that inclusionAI has built a strong small model, though the AA Bench score of 25 indicates there is still room for improvement on agentic tasks.
Local Tooling, Agents, and the Desktop Wave
Unsloth AI launched Unsloth Desktop, an open-source local AI tooling app that runs on Mac, Windows, and Linux. It supports MLX, GGUF, diffusion image and video, audio, CPU, and multi-GPU setups. It exposes OpenAI-compatible APIs and claims 2x faster training with 70% less VRAM. The app extends local AI beyond chat interfaces into a full operating environment, a sign that local tooling is maturing.
OpenAI released a ChatGPT desktop app for Linux in preview. It supports Ubuntu 24.04 and 26.04, Debian 13, Fedora 43 and 44, on both x64 and ARM64. The app can import and sync projects, chats, skills, and plugins from other agents into ChatGPT Work and Codex. OpenAI is clearly aiming to make its desktop integration a hub for agent workflows.
xAI introduced Grok Bot, an agent product that provides AI teammates with cloud computers. These bots can sign into tools, watch Slack threads and GitHub Actions, repeat scheduled routines, and create or manage other bots. The product hints at a "virtual coworker" category that goes beyond single-turn assistants.
Unsloth Desktop represents a maturation of local AI tooling. The app runs on Mac, Windows, and Linux and supports a wide range of formats, including MLX, GGUF, diffusion image and video, audio, CPU, and multi-GPU setups. The OpenAI-compatible APIs mean that existing applications can switch to local inference without major changes. The claims of 2x faster training with 70% less VRAM are significant for developers who want to fine-tune models on consumer hardware.
The ChatGPT desktop app for Linux is a notable move from OpenAI. The preview supports Ubuntu 24.04 and 26.04, Debian 13, Fedora 43 and 44, on both x64 and ARM64. The ability to import and sync projects, chats, skills, and plugins from other agents into ChatGPT Work and Codex positions the app as a hub for agent workflows. This is a direct challenge to the local tooling wave, as OpenAI tries to keep developers within its ecosystem.
Grok Bot from xAI is a stronger product signal than another model launch. The idea of AI teammates that can sign into tools, watch Slack threads and GitHub Actions, repeat scheduled routines, and create or manage other bots points to a future where agents are persistent workers rather than one-shot responders. The cloud computers that these bots run on suggest that xAI is building infrastructure for long-running agentic workloads.
Benchmarking, Evaluation, and Verifiable Inference
LlamaIndex launched ExtractBench, a benchmark with 370 documents, 4,869 pages, and 67 document types. The benchmark reveals a striking failure mode: commercial VLMs keep precision high, but recall collapses below 35% on documents longer than 50 pages due to silent truncation. That means models are quietly dropping content without any warning, a serious problem for document extraction workflows.
LlamaIndex also launched an "Agentic Plus" extraction tier. The company claims 95.6% value accuracy at less than one-third the cost of the nearest peer. If accurate, that is a significant cost-performance improvement.
Artificial Analysis released AA-AnalystAgent, a benchmark with 80 tasks using a pass^5 reliability metric. Claude Opus 5 leads at 54%, GPT-5.5 scores 50%, Claude Fable 5 scores 49%, and Kimi K3 is the top open-weights model at 39%. The gap between closed and open models remains, but it is narrowing.
Microsoft research found that feeding the previous hidden state at decoding time gives free gains. Compact natural-language skills distilled from trajectories recover 55% to more than 100% of the gap between non-reasoning and reasoning modes, with 2.7-6x fewer output tokens. That finding suggests that reasoning behavior can be captured in lightweight skills, making it available to smaller models without the full reasoning overhead.
Evaluation is shifting toward reliability and workflow correctness over one-shot capability. The pass^5 metric, the long-horizon tasks, and the domain-real benchmarks all point in the same direction. At the same time, benchmark skepticism is rising. Some observers argue that evals are being "vibed" rather than engineered, leading to broken scoring and exploitable sandboxes. That skepticism is healthy, but it also complicates the task of comparing models.
The ExtractBench findings are particularly concerning for enterprise users. Silent truncation means that a model may process only the first 50 pages of a document and then drop the rest without any warning. The model's precision stays high because the content it does extract is accurate, but the recall collapses because large portions of the document are never seen. For document extraction workflows, this is a critical failure mode. A legal team using a VLM to extract clauses from a 200-page contract would receive a partial result and have no way to know that content was missing.
The "Agentic Plus" extraction tier from LlamaIndex is a direct response to this problem. The claimed 95.6% value accuracy at less than one-third the cost of the nearest peer suggests that the tier is designed to handle long documents reliably. If the claims hold up, this would be a significant improvement for extraction workflows.
The AA-AnalystAgent benchmark from Artificial Analysis uses a pass^5 reliability metric, which measures whether a model can complete a task successfully across five attempts. This is a stricter standard than one-shot accuracy. Claude Opus 5 leads at 54%, followed by GPT-5.5 at 50% and Claude Fable 5 at 49%. Kimi K3 is the top open-weights model at 39%. The gap between closed and open models is narrowing, but it remains significant.
The Microsoft research on hidden state feeding is a fascinating result. By feeding the previous hidden state at decoding time, the researchers found that compact natural-language skills distilled from trajectories can recover 55% to more than 100% of the gap between non-reasoning and reasoning modes. This recovery comes with 2.7-6x fewer output tokens. The implication is that reasoning behavior can be captured in lightweight skills, making it available to smaller models without the full reasoning overhead. This could be a path toward more efficient reasoning models.
The rising benchmark skepticism is a healthy counterweight to the flood of new evaluations. Some observers argue that evals are being "vibed" rather than engineered, meaning that benchmarks are designed to make models look good rather than to measure real capability. This leads to broken scoring and exploitable sandboxes. The skepticism complicates the task of comparing models, but it also pushes the community toward more rigorous evaluation methods.
Attestable launched with $20 million in seed funding for zero-knowledge proofs for AI integrity. The startup aims to make verifiable inference a product rather than a research curiosity.
Vitalik Buterin, Ethereum co-founder, commented on the overhead estimates. He estimates that ZK overhead may be within single-digit, meaning less than 10x, relative to raw inference in some settings. That is a much smaller penalty than many assumed, and it makes verifiable inference practical for at least some workloads.
The move from theory to product is visible across the ecosystem. Deterministic integer-only inference, demonstrated across A100, H100, Apple M5 Max, AMD EPYC, and Intel Xeon, produces identical hashed logits. In a Qwen3-0.6B test, integer-only inference achieved a WikiText2 perplexity of 20.72 versus 20.95 for fp16, a negligible difference. It hit 106 tokens per second in CUDA-graphed decode on an A100 at batch 1, claimed to be 3.6x the fp16 eager baseline.
These claims may be robust, but they need verification. If they hold, deterministic inference has implications for reproducibility and proof-friendly inference. Software-generated per-chip adaptation could also weaken NVIDIA's CUDA moat, since portable inference kernels would reduce the advantage of proprietary hardware stacks.
Qdrant 1.19 added prefix matching on keyword indexes, a small but useful improvement for retrieval workloads. Together, IBM, and NVIDIA announced enterprise inference infrastructure on IBM Cloud, bringing together the three companies' strengths in serving, cloud, and hardware.
The AI news coverage for August 10-11, 2026, checked 12 subreddits, 544 Twitters, and no Discords. AINews is now a section of Latent Space, a consolidation that reflects the growing importance of daily AI journalism.
The security disclosure dominated the news cycle, but the broader story is the rapid maturation of the AI ecosystem. Models are getting smaller and faster. Evaluation is getting more rigorous. Verifiable inference is becoming a product. And the hidden reasoning traces that labs tried to protect are now exposed, with all the privacy and security implications that entails.
The watermarking debate continues as well. Critics question output bloat from watermarking schemes, while others argue that the entropy budget in model outputs allows subtle signatures without visible degradation. EU-style compliance pressure on AI text watermarking is adding urgency to that debate.
For now, the immediate concern is the security disclosure. The attack works, the secrets are real, and the fixes are partial. Labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces. Hidden chain-of-thought is not a reliable monitoring interface, and treating it as one is a mistake.
The disclosure also raises questions about distillation. Chinese labs have been accused of distillation, but the practice has not been prevented. This attack is not a scalable distillation path, according to the analysis, but it does lower the barrier for extracting specific reasoning traces. That is enough to worry model providers.
The 30B parameter tier is becoming crowded. Qwen, Google, NVIDIA, and Meta are all shipping models in that range within days of each other. The competition is pushing down costs and improving quality, but it is also creating a confusing landscape for buyers.
Grok Bot is a stronger product signal than another model launch. The idea of AI teammates that can sign into tools, watch Slack threads, and manage other bots points to a future where agents are persistent workers rather than one-shot responders. That is a meaningful shift.
The local AI tooling wave is expanding beyond chat interfaces. Unsloth Desktop, the ChatGPT Linux app, and the growing list of local models all point to a future where AI runs on consumer hardware. The Muse Glimmer 30B running on a single RTX 3090 is a concrete example of what is possible.
Verifiable inference is moving from theory to product. Attestable's $20 million seed round and Buterin's overhead estimates suggest that ZK proofs for AI integrity are close to practical. That could change how we trust model outputs.
The security disclosure is the story of the week, but it is not the only story. The ecosystem is moving fast, and the pace shows no sign of slowing.