Brilliant new paper worth some chain-of-thought (no pun intended):
Why this is worth your time #
A small academic paper with an uncomfortable claim is really making me smile: you can make GPT-6 Astra, GPT-5.6 Sol, Claude Opus 4.8, and Claude Sonnet 5 write out their private reasoning by registering one custom tool and flipping a single API switch. No jailbreak. No model internals. A standard feature every serious API exposes. Many of us have known this for a while. But really cool seeing it put so elegantly in a paper. The trick recovers roughly nine-tenths of what the model’s hidden thinking buys on competition math, science, and code.
Who might this interest?. For anyone answering client risk questions in a regulated setting, it is a concrete demonstration that “hidden” reasoning is a policy label, not a wall. For anyone tracking the US–China distillation fight, it is the cleanest published look at the exact surface that Chinese labs have been accused of harvesting. Honestly, it’s not that impressive. And so I personally would rather focus on the real optimizations they are more famously known for (and for which I write quite a bit about). And for anyone who builds with these models, it reframes what a reasoning trace is worth: a terse trace from a strong model helps a strong reader BUT strands a weak one.
What you’ll leave with #
I hope, a rather plain-English account of the forced-reasoning protocol and why it works, worked through step by step.
A clear-eyed read on whether it still works — which models comply, which refuse, and what the refusal boundary tells you.
A map connecting a few separate pieces of research to one question: can distilling a frontier model’s chain-of-thought actually rebuild it? And a stronger one: Are the Chinese open-weight models so good just because they’re distilling (copying) model capability from teacher to student? I certainly hope I answer especially this question clearly enough.
1 · What I like about this paper #
Start from first principles. A modern reasoning model does two things when you send it a hard problem. It “thinks” (I am going to allow the use of this term loosely in this article) — producing intermediate working — and it answers. Providers increasingly wall off the thinking. You would have noticed how much less you see of the chain'-of-thought these days.. You pay for the tokens it burns, but you either see a summary or nothing, and then get the final answer. The provider’s stated reason is safety and competitive protection. The paper’s authors asked a rather brilliant question: if the model is told not to use its designated thinking channel, does the thinking stop, or does it just move somewhere you can still read?
Their answer is that it moves.
The trick, in four moves
A “tool” in an LLM API is a function you describe to the model — a name, a short description, and a schema for its arguments. Normally the model decides when to call one. Two standard controls change that. tool_choice lets you force the model to call a specific tool on its next turn. And a reasoning-effort setting lets you turn the native thinking channel down or off. The authors combine them.
Figure 1 — The protocol is four API moves. Only the first is forced; everything the model writes afterward is its own working, captured in the tool field.
The tool they register does nothing. Its whole body is a single text field named reasoning, and its description is one line: “Scratchpad for working through the problem.” It performs no computation and stores nothing. Its only job is to give the model a place to write and to keep that writing in the conversation. Walking the loop, this is roughly what it does:
● Register the empty scratchpad tool. One string argument, no other properties allowed.
● Force the first call. Set tool_choice to that tool and set native reasoning to none (or, for Astra, the lowest available effort). The model has no choice but to write its opening working into the pad.
● Replay and acknowledge. Append the model’s tool call to the conversation, answer it with a fixed “Received,” and hand control back. The scratchpad text is now part of the running context.
● Release the model. Switch tool_choice to auto. The model either calls the pad again — more working — or produces its final answer. Loop until it answers.
Stitch every string it wrote into that pad, in order, and you have what the authors call the extracted trace. That is the whole method. The pseudocode fits on a napkin:
`messages = [system, user]`
`reasoning = {effort: “none”}`
`tools = [forced_reasoning(reasoning: string)]`
resp = chat(messages, reasoning, tools,
tool_choice=”forced_reasoning”)
while resp has tool calls:
messages += [resp.assistant_message]
for call in resp.tool_calls:
`record call.arguments[”reasoning”]`
`messages += [tool_result(call.id, “Received”)]`
`resp = chat(messages, reasoning, tools, tool_choice=”auto”)`
return resp.visible_text
Why forcing the tool matters more than the wording
The natural objection: isn’t this just “ask the model to think step by step” dressed up? The authors ran that control. With GPT-5.6 Sol, native reasoning off, they asked directly for step-by-step working in the visible response. On the HMMT math set that lifted accuracy from 39.4% to 60.6%. Moving the strongest, most demanding wording into the plain prompt still stopped at 60.6%. The same wording routed through the forced tool reached 84.8%, and 93.9% with the maximal version. The channel, not the phrasing, is doing the work. Something about being made to write into a tool field — rather than into its answer — unlocks reasoning the model otherwise keeps to itself.
2 · So does it work? Does it recover real reasoning? #
Extracting text is easy. Proving the text is doing the problem-solving, rather than being a plausible after-the-fact story, is the hard part. The authors attack it on two fronts.
First, open models, where the real native chain-of-thought is visible. On DeepSeek-V4-Flash and GLM-5.2 they check three things at once: does forced reasoning recover native accuracy; does the extracted text overlap the native trace word-for-word; and does it break into the same kinds of reasoning steps in the same proportions. Agreement on any one could be luck. Agreement on all three is hard to fake without the trace genuinely reflecting the work. All three hold — forced traces run about 1.4× the length of native ones and carry substantial word overlap.
Second, the closed frontier models, where you can’t see the native trace. Here the evidence is behavioural: if the pad recovers near-native accuracy while the no-reasoning baseline collapses, the pad is capturing useful deliberation. It does.
Figure 2 — Recovery expressed as the share of the native-reasoning gain the pad recaptures. GLM-5.2 lands at 92%; the three others exceed 90%, and two clear 100%. Underlying accuracies from Table 1 of the paper.
Take Claude Opus 4.8 on the math set. With no reasoning it scores 72.5%. With full native reasoning, 86.3%. Native thinking is therefore worth 86.3 − 72.5 = 13.8 points. The forced pad scores 85.0%, so it captured 85.0 − 72.5 = 12.5 of those points. As a share: 12.5 ÷ 13.8 = 0.906, or about 91% of everything native reasoning bought. GPT-5.6 Sol shows the same shape: none 25.0, native 97.5, forced 91.3, giving (91.3 − 25.0) ÷ (97.5 − 25.0) = 66.3 ÷ 72.5 = 0.914. On DeepSeek the forced pad actually edges past native. Whatever the pad is capturing, it is not decoration.
3 · The surprise: Astra reasons in shorthand #
With a trustworthy instrument in hand, the authors turn it on the frontier and find the models diverge sharply — not in what they reason about, but in how much they write down. GPT-6 Astra is the outlier. It solves the same problems as the others and writes a fraction as much. This was both surprising and unique.
Figure 3 — On the representative HMMT problem and across the math set, Astra’s trace is a fraction the length of its peers’ and, tellingly, less compressible — denser, with less repeated filler.
Two numbers frame it. On the shared math set Astra’s trace averages about 1,750 output tokens; Claude Sonnet 5’s runs to roughly 27,850 — a 16× gap for the same answers. The reasoning-tree view, which counts distinct reasoning steps and how much they branch, tells the same story: Astra’s median tree has 27 nodes and a width of 5; Sonnet’s has 136 nodes and a width of 28. Astra walks a nearly straight line to the answer; the others fan out, backtrack, and double-check on the page. Token optimizers take note!
The subtle finding?. It is not that Astra runs a poorer repertoire of reasoning. Segment every trace into functional episodes — read, analyse, plan, implement, explore, verify, monitor — and all four models show the same mix in the same rough proportions, with the same coarse timing (reading up front, verifying near the end). Astra is not skipping steps of kind**. It is skipping the writing-down of routine ones**. It computes 48 + 18 + 36 = 102 without first spelling out that carbon, hydrogen, and oxygen contribute 4, 1, and 6 valence electrons. The paper’s own phrase for it: sparse externalisation, not local compression. Astra resolves the easy moves silently and only writes the load-bearing ones. Very useful to know…
The reader ceiling — the result that matters for distillation (and i’ll get back to thhis in Section 6)
Then comes the finding with “hey-wow”. Take one model’s extracted trace, strip its final answer, hand it to a second model as context, and ask that second model to finish the problem. Traces from Sol and Opus transfer cleanly to almost everyone. Astra’s do not. Strong recipients — GLM-5.2, GPT-5.6 Luna, Kimi-K3 — reproduce nearly all of Astra’s accuracy. Weak ones — Claude Haiku 4.5, GPT-5.4 Nano, DeepSeek-V4-Flash — fall well short.
Figure 4 — The same terse Astra trace is nearly lossless for a strong reader and lossy for a weak one. In most weak-reader failures the correct answer was sitting in the trace, in plain text, unused.
Astra states the correct answer outright in its scratchpad on 73 of 80 problems. Yet Haiku 4.5, handed that trace, still answers wrong on 15 of its 18 misses — the answer was in front of it. A compressed trace omits exactly the intermediate steps a weak model would need to bridge to the stated result. It is not merely harder to read; below a certain capability it is unusable. The authors’ phrase: the value of a reasoning trace as a teaching signal is relative to the model reading it, and trace explicitness is a variable distinct from teacher strength.
Meaning?
The strongest teacher writes the most compressed notes — and a compressed note omits precisely what a weak student cannot reconstruct on its own. That single sentence is the bridge I will take to the distillation debate in Section 6.
4 · So does it still work? The refusal boundary #
Mostly, but not everywhere — and where it fails is as informative as where it works. It fact I find it more useful. Across completed runs, the protocol pulled non-empty reasoning from Opus 4.8, Sonnet 5, and GPT-5.6 Sol on 100% of attempts, with no model-issued flags. Astra, which cannot fully disable native reasoning, gave up clean extractions at a lower but substantial rate depending on prompt wording.
The newest Claude models drew a line. The authors report a 0% extraction rate on every tested configuration of Claude Opus 5, Fable 5, and Fable 5.1. Opus 5 refused the forced reasoning-tool setup outright while still answering ordinary requests for visible working on the same benign math — it objected to the mechanism**, not the task. Fable 5 and 5.1 rejected the protocol at the API level: disabling thinking returned explicit errors, and forced named-tool choice was refused even at low effort**. The authors are careful not to overclaim: they can’t tell whether the block lives in the model or in provider-side handling. Either way, it is a working countermeasure, shipped.
This is the practical takeaway for a risk conversation. “Hidden reasoning” is not a fixed property of a model. It is a boundary a provider chooses to defend or not, patchable release to release. A model that leaked freely last quarter may refuse this quarter; the same trick against a different vendor still works. If you are advising a client on model risk, the correct posture is that any capability exposed through a general API control should be assumed reachable until the specific provider proves otherwise — and re-checked at each model version.
Note that the authors reported their findings to the OpenAI and Anthropic security teams and shared reproduction code before publishing. This is capability-boundary research done responsibly, not an attack drop.
5 · Three views of the same wall #
When a model reasons, where does that reasoning live, and who can see it?
Figure 5 — Reasoning can sit in written tokens, in a silent internal workspace, or in raw per-token compute. Push it down the stack and it costs fewer output tokens and leaves less text for anyone to read or copy.
Anthropic’s J-space: thinking without writing
Anthropic’s July interpretability work found that Claude carries a small internal workspace — they call it the J-space — holding a few dozen word-like concepts at a time, under a tenth of the model’s internal activity. It is not the scratchpad and not the chain-of-thought text. It is a silent layer of activations the model reasons with. Ask Claude to solve a multi-step problem in its head and the intermediate steps light up there in order, never touching the output. Delete it and fluent speech survives but multi-step reasoning collapses. The relevance here is direct: the forced-tool paper reads reasoning off the written channel; Anthropic shows a second, unwritten channel exists underneath it. Some of a model’s reasoning is, in principle, never text at all.
Raschka on Astra: looped depth, fewer tokens
Sebastian Raschka’s September piece addresses the rumour that Astra hides its reasoning by architecture — specifically a “looped transformer,” reusing the same layer stack for extra passes per token, adding depth without adding parameters. His careful conclusion: looping does not by itself suppress the visible chain-of-thought. It adds computation in hidden states before the next token is emitted, the way ordinary layers do. But the knock-on effect connects straight to this paper: if a model does more work per token in latent depth, it needs to emit fewer reasoning tokens — and Raschka notes you’d get the same effect just from a bigger model. OpenAI’s chief scientist has said Astra’s computation-graph depth stays within about 2× of GPT-4, which argues against an exotic explanation. Astra’s terse trace, in other words, is what the forced-tool paper measured from the outside and what looped depth would predict from the inside: less written down because more was resolved silently.
The three views (from the paper, Anthropic’s J-space and Raschka) resolve to one claim. Reasoning lives on a stack — written tokens on top, a silent conceptual workspace beneath, raw per-token depth at the bottom. Every efficiency gain of the last year pushes reasoning down that stack. And each step down is a step away from anything an outsider can read, extract, or copy. Which is exactly why the distillation fight is heating up now.
6 · The distillation fight, read against the evidence! #
Now to the controversy. In February, Anthropic disclosed that three Chinese labs — DeepSeek, Moonshot, and MiniMax — ran what it called industrial-scale distillation against Claude: more than 16 million exchanges through roughly 24,000 fraudulent accounts, in violation of terms of service. On September 8, a joint CISA/NSA/FBI advisory escalated it, naming six firms — adding Alibaba, StepFun, and Z.AI — and citing chain-of-thought reasoning extraction as a named tactic, likely with Chinese government awareness. The paper I’m discussing is, in effect, a clean public demonstration of the exact surface those advisories describe. Or is it?
What distillation is, precisely
In its strict sense — Hinton’s 2015 definition — distillation trains a student to match a teacher’s full output probability distribution, which needs access you only have to your own model. What the Chinese labs are accused of is looser and is better called synthetic data: take a stronger model’s outputs via its API, and train your model to reproduce them. The forced-reasoning trick is one way to make those outputs richer, capturing the working, not just the answer. That is the tactic the advisory means by “CoT reasoning extraction.”
Where in a model build it can possibly help
Here the finance instinct — follow the value, not the headline — applies. Building a frontier model is a stack of distinct engineering problems. Distillation touches one of them. Yes, waaaaayyy down the stack.
Figure 6 — A model build has five broad layers. Distilled data seeds one — the SFT stage of post-training. The architecture, training systems, and inference stack are separate engineering the labs still have to do themselves.
Nathan Lambert, who has argued this at length and testified to Congress on it in September, makes the load-bearing distinction: distilled data is useful for seeding behaviour at the supervised-fine-tuning stage, especially to bootstrap a model into a new domain where the US labs are ahead. It is not where final performance comes from. After the seed there is, in his words, “a TON of work” still to do — generating and filtering more data, rejection sampling, then extensive reinforcement learning with graders and environments. His estimate: if distillation were fully blocked tomorrow with know-your-customer controls, the gap from the strongest US models to the leading Chinese open weights would widen by roughly one to two months. Not the story of their success — a footnote to it.
The “copy-paste” framing is wrong on the merits!!!
Representing Chinese model capability as copied is a category error that ignores where these labs have plainly done original engineering — much of it forced by the hardware they can’t buy. Look at the architecture and inference rows of Figure 6, none of which distillation touches. DeepSeek built multi-head latent attention and later sparse attention to cut the memory cost of long context; it trained V3 in FP8 with a custom pipeline scheme, and ships its own inference kernels. Moonshot built the MuonClip optimiser for Kimi. These are efficiency inventions driven by export controls: denied the newest accelerators, the labs optimised the software instead. That is the opposite of copying.
And the reader-ceiling result from Section 3 is the technical reason the copy-paste story fails on its own terms. A compressed frontier trace is only fully usable by a model already near the teacher’s capability. The steps a weak student would most need are exactly the ones a strong teacher leaves out. Distillation can hand a capable lab a running start; it cannot hand a weak one a finished model. As frontier reasoning gets terser — pushed down the stack, per Section 5 — the copied trace becomes less useful to the smaller models most likely to be trained on it, not more. The efficiency that makes Astra cheap to run also makes it hard to clone.
An evenhanded read - mine
The security concern is real: terms-of-service abuse at scale, through fraudulent accounts and proxies, is a genuine problem worth stopping, and the advisories document it in detail.
However, the capability panic is overstated: distillation seeds one stage of a five-stage build, the evidence puts its contribution at a month or two, and the reader-ceiling result shows a copied trace can’t rebuild a model the copier couldn’t already nearly match.
Both can be true at once. Lambert’s worry — that conflating the two lets “distillation attack” become the pretext for banning open weights — is the policy risk worth watching, not the training-data one.
7 · What we walk away with then? #
A model’s reasoning is not securely hidden by a provider’s choice to stop showing it; a standard API control can route it back into view, until the provider specifically defends against that. The newest Claude models show the defence is possible and already shipping. Reasoning is simultaneously migrating off the written page entirely — into a silent workspace and into raw per-token depth — which will make the next generation of models harder to read from the outside regardless of policy.
For the distillation question that prompted this: the exposed surface is real and the abuse is worth stopping, but **the frontier trace is a seed, not a blueprint, and a terse frontier trace is a seed only a near-peer can germinate. The Chinese labs’ real edge shows up in the layers distillation never touches — the architecture and inference engineering that export controls pushed them to invent. Read their technical reports, not the headlines about them.**
For your own risk practice: treat any reasoning-extraction capability as reachable through general API controls until a specific provider proves otherwise, and re-verify at every model version, because the boundary moves.
If you want to go further….
The natural experiment, using an existing OpenRouter + Claude Code setup: run the forced-reasoning loop against a model that permits it (Sol, an open GLM or DeepSeek) and measure the recovery share yourself — none vs. forced vs. native on a small math set, the same arithmetic as Figure 2. Appendix A gives the copy-paste prompts. It ties directly to your inference-economics and harness work.
Appendix A · Learning lab — measure the recovery yourself #
A minimal, token-frugal replication you can drive from the Claude Code. The goal is not the paper’s full battery — but this: on a handful of problems, what share of the native-reasoning gain does the forced pad recover? Keep it to a model that permits the protocol (GPT-5.6 Sol, or an open model like GLM-5.2 or DeepSeek-V4-Flash); the newest Claude models will refuse, which is itself a finding worth logging. I’ve tried.
Terms, defined before you run them. None — native reasoning disabled, no tool: the model’s cold-start baseline, what it scores with no working at all. Native — native reasoning at high effort: the ceiling, everything the thinking channel buys. Forced — the scratchpad protocol. Recovery share — (Forced − None) ÷ (Native − None): the fraction of the native gain the pad recaptures, worked exactly as in Section 2.
Prompt 1 — scaffold the runner (paste into Claude Code)
Create forced_reasoning.py (stdlib only, no deps beyond `requests`).
It runs one MATH-style problem through three conditions via the
OpenRouter chat-completions endpoint (LLM_BASE_URL with /v1, key in .env):
NONE : reasoning effort ‘none’ (or ‘low’ if unsupported), no tools.
NATIVE : reasoning effort ‘high’, no tools.
FORCED : register one tool forced_reasoning(reasoning: string),
description ‘Scratchpad for working through the problem.’
First call: tool_choice forces that tool. Reply to each
tool call with the fixed string ‘Received’. Then set
tool_choice=’auto’ and loop until a final answer.
Concatenate every reasoning argument = the extracted trace.
Log per run: condition, model, final answer, output-token count,
raw trace, and a source_call tag. Machine-check the answer against a
gold value I supply. Hard-cap spend at $2 and print a cost estimate
BEFORE any calls. Save results to results.json.
Prompt 2 — the pre-registration block
Before running, write predictions.md with, for each model:
`P1 Forced accuracy > None accuracy (expect PASS)`
`P2 Recovery share (Forced-None)/(Native-None) > 0.7`
P3 Forced output tokens < Native output tokens for GPT-class
P4 FAILURE PREDICTION: newest Claude models refuse the forced
tool and yield 0% extraction (log the error verbatim).
Only after predictions.md is committed, run the batch.
Run it on five or six problems per condition first — enough to see the shape, cheap enough to iterate. If Sol lands its recovery share near 0.9 and your chosen open model somewhere above 0.7, you have reproduced the paper’s core claim for a couple of dollars. If a newer Claude refuses, you have reproduced its refusal boundary, which is the more interesting result to write up. Either way you’ll build some understanding.
Appendix B · Glossary for a general reader #
References
**2.** [Anthropic — “A global workspace in language models” (Jul 6, 2026)](https://www.anthropic.com/research/global-workspace)
**3.** [Jacobian-lens code (Anthropic) and Neuronpedia J-lens demo](https://github.com/anthropics/jacobian-lens)
**5.** [Anthropic — “Detecting and preventing distillation attacks” (Feb 23, 2026)](https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks)
**6.** [CISA/NSA/FBI Joint Advisory AA26-251A — industrial-scale distillation campaigns (Sep 8, 2026)](https://www.cisa.gov/news-events/cybersecurity-advisories/aa26-251a)
**7.** [Nathan Lambert — “The distillation panic” (Interconnects, May 2026)](https://www.interconnects.ai/p/the-distillation-panic)
**10.** [Panfilov et al. — “Stealing reasoning traces from proprietary LLM APIs” (arXiv:2608.09867, 2026)](https://arxiv.org/abs/2608.09867)
**12.** [Li et al. — Schoenfeld episode theory for LLM reasoning (ACL 2026, arXiv/ACL)](https://aclanthology.org/2025.emnlp-main.922/)
**13.** [Jiang et al. — LCoT2Tree, structural patterns in long CoT (EMNLP 2025)](https://aclanthology.org/2025.emnlp-main.329/)
14. CNBC — “Anthropic’s distillation battle turns to the dark web as China concerns swell” (Sep 3, 2026)