In Part 1 of our series, we saw why non-thinking language models fail the classic "strawberry" test (0 / 64 correct when asked for a single number)—and how flipping the Thinking switch immediately takes Qwen3-4B to 256 / 256 (100% accuracy).
So for Part 2, we opened up the <think> scratchpad and watched the model think token-by-token.
Here is a real, unedited trace we captured from Qwen3-4B (temperature 0.6, seed 1003) while it was counting the R's in "strawberry":
Look closely at what happened:
Nobody wrote an if mistake: print("Wait") rule in the code. Nobody programmed the model to doubt itself.
So where does that “Wait” come from? What does the “Thinking” button actually do at the token level? How does a neural network learn to second-guess its own output — and when does extended thinking backfire?
We ran 696+ benchmark completions, probed internal layer entropies and attention heads, measured GPU KV-cache memory to the byte, and ran our own GRPO reinforcement learning training loop on a GPU to find out.
🎥 Prefer watching the visual engineering breakdown? Watch our full 14-minute animated video here:
The Misconception: “Turning on Thinking Mode switches the API from a fast chat model to a separate, heavier reasoning model.”
In hybrid reasoning models like Qwen3–4B, both modes use the exact same model file and the exact same 4,022,468,096 parameters across 36 transformer layers.
Here is the entire difference between enable_thinking=True and enable_thinking=False at the prompt template level:
That’s it! It hands the model an empty <think>...</think> box that is already welded shut. Seeing that the </think> closing tag has already passed, the next-token predictor skips straight to generating the final user-facing answer.
Takeaway #1: Thinking Mode isn’t a second brain bolted onto the model. It is the exact same next-token predictor — except the prompt template hasn’t pre-closed the <think> tag.
Why does writing inside <think> rescue accuracy on "strawberry"?
In Episode 1, we saw that " strawberry" is a single integer token (#72600 in Qwen3; #101830 in GPT-4o). The model cannot inspect the individual letters inside a single token ID. But when the model spells S - T - R - A - W - B - E - R - R - Y inside its <think> block, each letter becomes its own distinct token in the context window.
We proved this with three experiments:
At every single token position t inside the cold-open <think> trace, we appended a forced </think>\nFinal Answer: suffix and measured the model's exact probability P("3") if it were forced to answer at that exact millisecond:
What if the model just benefits from running extra forward passes (“extra test-time compute”), regardless of what words it writes?
We took 107 real thinking traces and replaced every thinking token inside <think> with the exact same number of period dots (. . . . .), giving the transformer the exact same number of forward passes and attention columns:
Extra forward passes with dots do almost nothing (35.7% vs 33.6%). The semantic content written onto the scratchpad is the working memory.
When Qwen3-4B finally closes </think> and outputs "3", where do its attention heads look?
Is the word "Wait" just random temperature noise? To find out, we measured the Shannon Entropy (uncertainty across the vocabulary distribution, in bits) at every token in the trace:
H(X) = − Σᵢ pᵢ · log₂(pᵢ)
Across normal calculation tokens inside <think>, the model is near-deterministic: the median entropy is just 0.012 bits.
Right at discourse markers like "Wait", "Hmm", and "But", however, token entropy spikes 30× to 200× higher—between 0.34 and 2.90 bits (the 65th to 99.7th percentile of uncertainty). Recent interpretability research calls these high-entropy pivots "Forking Tokens" (arXiv:2506.01939): the sparse branching points where a reasoning trajectory can change direction.
And when we ran our Logit Lens across all 36 layers at the exact moment "Wait" was generated, "Wait" did not become the #1 top candidate until Layer 33 of 36. Self-doubt is computed in the very final layers of the transformer.
We tested Qwen3-4B across multiple tasks and token budgets—and uncovered a nuanced reality that hype threads miss.
On 100 GSM8K grade-school math problems (8 trials each, 800 completions):
Notice that! When the model is allowed to show its work out loud in the normal response, 90.6% vs 93.9% is within statistical noise. Similarly, on "strawberry", if you force a bare number with Thinking OFF, it gets 0 / 64 (0%); with Thinking ON, 256 / 256 (100%); and with Thinking OFF without the bare-number restriction, it spells the word out in the main reply and gets 20 / 20 (100%).
Where <think> truly shines is on frontier competition problems where deep search and backtracking are required: on AIME 2024 (Reported in the Qwen3 Technical Report, Tables 17–18), Qwen3-4B jumps from 25.0% (Thinking OFF) to 73.8% (Thinking ON).
More thinking is not free — and on simple or deceptive problems, it can actively hurt:
On our 100 GSM8K problems, Thinking Mode generated 1.84 million tokens compared to 0.24 million tokens with Thinking OFF — 7.7× more tokens and 6.6× higher median latency (25.9 seconds vs 3.9 seconds).
Even though the user UI hides the <think> block behind a collapsible dropdown, the GPU cannot forget a single thinking token while generating the response. Because every future token attends back to every past token, the GPU must store the **Key (**K) and **Value (**V) vectors for every token across every layer in high-bandwidth VRAM (the KV-Cache).
In Qwen3-4B (bf16 = 2 bytes per parameter, 36 layers, 8 GQA key-value heads, head_dim = 128), the exact memory footprint per token is:
KV bytes per token = 2 (K,V) × 36 layers × 8 KV heads × 128 dim × 2 bytes = 147,456 bytes ≈ 144 KB
Why couldn’t we just prompt "Let's think step by step" like researchers did in 2022 (Kojima et al., 2022; Wei et al., 2022)? Because as Google's original Chain-of-Thought paper proved, zero-shot CoT "only yields performance gains when used with models of ~100B parameters"—smaller models produced fluent but illogical chains of thought.
Today, a 4-billion-parameter model reasons effortlessly because of two training breakthroughs:
Instead of paying humans to grade every intermediate line of math, DeepSeek-R1 (arXiv:2501.12948) showed that you can give a model problems with automatically verifiable final answers (math, code) and train it via GRPO (Shao et al., 2024):
Aᵢ = (rᵢ − mean(r₁, …, r_G)) / std(r₁, …, r_G)
On a real GSM8K problem from our run where the true answer was 16, the model sampled [16, 4, 16, 16, 16, 4, 16, 4] → rewards [1, 0, 1, 1, 1, 0, 1, 0] (mean = 0.625, std = 0.5175). Every token inside the 5 correct traces got a positive nudge of +0.725, while every token inside the 3 wrong traces got pushed down by -1.208.
To see this live, we trained Qwen3-0.6B-Base (a tiny 600M-parameter base model) for 250 steps of GRPO (~90 minutes on a single GPU):
In DeepSeek-R1’s 671B-parameter RL run, reflective words like "Wait" spiked 5- to 7-fold during training. Yet in our tiny 0.6B GRPO run, "Wait" frequency actually dropped (0.004 → 0) as the small model locked onto a rigid "let's break it down step by step" template.
Recent studies (arXiv:2503.20783) show that RL amplifies self-correction behaviors that already exist in large base models (671B / 235B).
And how did our Qwen3-4B model get so good at saying "Wait"? As documented in the Qwen3 Technical Report (arXiv:2505.09388), Qwen first trained its massive 235B and 32B teacher models with RL—and then trained the compact Qwen3-4B student via Strong-to-Weak Distillation, aligning its token-by-token logits directly with the teacher's probability distributions!
Why AI Says “Wait”: How Thinking Models Actually Work (Every Step Measured) was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.