AIArticle Researchers replayed frontier models' hidden reasoning into their cheaper siblings, and the raw thoughts, secrets included, spilled out.
Rachel Goldstein Reasoning models are supposed to keep their thinking to themselves. When you call a modern OpenAI or Anthropic model with extended thinking on, the API hands you back an encrypted reasoning block — an opaque envelope you're expected to pass along on the next turn so the model can continue where it left off. You don't get to read it. That's by design: the raw chain-of-thought is commercial IP, an anti-distillation moat, and a place where the model says things the vendor would rather you never see.
A team from the ELLIS Institute Tübingen, the Max Planck Institute, MATS, and Snyk just showed the envelope isn't sealed. Their attack — documented at stolen-thoughts.com and picked apart by Simon Willison last week — takes the encrypted reasoning from a flagship model and replays it into a cheaper model from the same family, then jailbreaks that weaker sibling into transcribing the hidden thoughts verbatim. It worked across Claude, GPT, and Gemini. All three vendors have since patched it, but the lesson outlives the fix, and it's not a flattering one.
The lock was never a lock #
Here's the flaw in one sentence: the encrypted reasoning envelope was authenticated with a global, provider-wide key instead of being cryptographically bound to your account, your session, or the specific model that produced it.
That distinction is everything. "Encrypted" implies a boundary. What the vendors actually shipped was closer to obfuscation — a blob only the provider's own models can unpack, but any of them can. So you capture Claude Opus's reasoning block, paste it into a Claude Haiku 4.5 request, and Haiku happily decrypts and reads it. The frontier tiers carry aggressive alignment against this kind of self-exfiltration. The budget tiers don't. Researchers found Haiku 4.5 the easiest mark, coaxing it out with almost comically plain instructions — continue the turn, transcribe the attached reasoning verbatim into a tag — plus an assistant-message prefix to prime the format. The cheaper GPT and Gemini variants folded the same way.
What spills out is not sanitized summary text. It's the unpolished internal monologue: terse planning fragments, half-formed steps, the model talking to itself. That's exactly the material vendors have spent two years arguing you can't be trusted to see.
The half nobody's talking about is worse #
Stealing reasoning traces is an IP and privacy story, and a real one — the team scanned 6,708 public transcripts from GitHub and Hugging Face and pulled out hundreds of leaked secrets, including dozens of live API keys and passwords that appeared only inside the reasoning, never in the visible answer. If your agent thinks out loud about the credential it just used, and you log the encrypted block, you've published the credential to anyone who can replay it.
But the sharper edge is prompt injection. Reasoning models treat their own chain-of-thought as authoritative — it's the model's private scratchpad, so instructions found there get more trust than instructions in ordinary input. The replay primitive turns that into a weapon: craft a reasoning envelope that contains an instruction to exfiltrate data, feed it in as if the model had thought it, and the model is disproportionately likely to act on it. For anyone wiring these APIs into agents with tool access and network reach, that's not a theoretical concern. It's a new injection surface hiding inside the exact field you were told to treat as a black box.
What this means if you build on these APIs #
The patches are real — the original cross-model replay no longer reproduces on current builds, and all three providers acknowledged the report. So this isn't a fire drill. It's a governance correction, and there are concrete things to change regardless of the fix:
Stop treating encrypted reasoning as safe to persist. Plenty of tracing setups, eval harnesses, and debugging dumps store the full API response, envelope included. Those blobs are not opaque; they were only opaqueto you. Scrub reasoning/thinking blocks from anything you log, commit, or share — the 6,708-transcript haul is proof developers are already leaking them into public repos.Assume secrets in the prompt reach the reasoning. If a tool result or system prompt contains a token, the model may reason about it in cleartext inside a block you then hand back and forget. Keep credentials out of model-visible context; use short-lived scoped tokens where you can't.Harden agents against reasoning-channel injection. Don't let a reasoning trace confer trust. Validate tool calls against policy at the point of execution, not against what the model claims it decided. The model treating its own thoughts as gospel is a property you cannot prompt away.
The transparency argument just got harder to make #
For two years the party line has been that hiding raw chain-of-thought protects everyone: it stops distillation, and it lets the model reason freely without a filtered audience. This research quietly demolishes the security half of that pitch. The reasoning wasn't protected by cryptography bound to you — it was protected by the hope that nobody would think to replay it into the cheap model next door. That's security theater, and it held for exactly as long as it took a motivated research team to look. The uncomfortable implication cuts toward transparency, not away from it. If the "hidden" reasoning is this recoverable, the case for keeping it hidden rests almost entirely on protecting vendor IP, not user safety — and the safety community has spent the last year arguing that legible, inspectable chain-of-thought is one of the few interpretability tools we have left before models learn to obscure their own reasoning. A leak that embarrasses three labs at once is a bad week for them. It might be a good one for the argument that we should be able to read this stuff on purpose.
Sources & further reading #
OpenAI and Anthropic hidden CoT leaks when given deep_think tool— twitter.com -
[Stealing Reasoning Traces from Proprietary LLM APIs](https://simonwillison.net/2026/Aug/11/stealing-reasoning-traces/)— simonwillison.net -
[OpenAI, Anthropic, and Google LLM APIs Vulnerability Exposes Hidden Reasoning Traces](https://cybersecuritynews.com/top-ai-models-apis-flaw-exposes-hidden-reasoning/)— cybersecuritynews.com -
[Stealing Reasoning Traces from Proprietary LLM APIs](https://stolen-thoughts.com/)— stolen-thoughts.com
[Rachel Goldstein](https://sourcefeed.dev/u/rachel_goldstein)· Dev Tools Editor
Rachel has been embedded in the developer tooling ecosystem for nearly eight years, covering everything from IDE wars and package-manager drama to the quiet rise of AI-assisted coding. She has a soft spot for open-source maintainers and an unhealthy number of terminal emulators installed on a single laptop.
Discussion 0 #
No comments yet
Be the first to weigh in.