Why AI Says “Wait”: How Thinking Models Actually Work (Every Step Measured)

A token-by-token analysis of Qwen3-4B found that enabling Thinking Mode injects only 4 tokens ([151667, 271, 151668, 271]) that pre-close the <think> tag, while both modes run the same 4,022,468,096 parameters across 36 transformer layers. In experiments spanning 696+ benchmark completions, replacing thinking tokens with equal-length runs of periods raised accuracy only from 33.6% to 35.7%, whereas Thinking Mode took Qwen3-4B from 0/64 to 256/256 on the strawberry letter-counting test, indicating the semantic content of the scratchpad, not extra forward passes, drives the gain. Median token entropy inside <think> measured 0.012 bits but spiked 30× to 200× (0.34 to 2.90 bits) at discourse markers such as "Wait," "Hmm" and "But.

In Part 1 of our series https://youtu.be/7HBtSz3wSSM , we saw why non-thinking language models fail the classic "strawberry" test 0 / 64 correct when asked for a single number —and how flipping the Thinking switch immediately takes Qwen3-4B to 256 / 256 100% accuracy . So for Part 2, we opened up the <think scratchpad and watched the model think token-by-token. Here is a real, unedited trace we captured from Qwen3-4B temperature 0.6, seed 1003 while it was counting the R's in "strawberry": Look closely at what happened: Nobody wrote an if mistake: print "Wait" rule in the code. Nobody programmed the model to doubt itself. So where does that “Wait” come from? What does the “Thinking” button actually do at the token level? How does a neural network learn to second-guess its own output — and when does extended thinking backfire? We ran 696+ benchmark completions , probed internal layer entropies and attention heads, measured GPU KV-cache memory to the byte, and ran our own GRPO reinforcement learning training loop on a GPU to find out. 🎥 Prefer watching the visual engineering breakdown? Watch our full 14-minute animated video here: The Misconception: “Turning on Thinking Mode switches the API from a fast chat model to a separate, heavier reasoning model.” In hybrid reasoning models like Qwen3–4B , both modes use the exact same model file and the exact same 4,022,468,096 parameters across 36 transformer layers. Here is the entire difference between enable thinking=True and enable thinking=False at the prompt template level: The 4 tokens injected when enable thinking=False: "<think \n\n</think \n\n"token ids = 151667, 271, 151668, 271 That’s it It hands the model an empty <think ...</think box that is already welded shut . Seeing that the </think closing tag has already passed, the next-token predictor skips straight to generating the final user-facing answer. Takeaway 1: Thinking Mode isn’t a second brain bolted onto the model. It is the exact same next-token predictor — except the prompt template hasn’t pre-closed the <think tag. Why does writing inside <think rescue accuracy on "strawberry"? In Episode 1 https://youtu.be/7HBtSz3wSSM , we saw that " strawberry" is a single integer token 72600 in Qwen3; 101830 in GPT-4o . The model cannot inspect the individual letters inside a single token ID. But when the model spells S - T - R - A - W - B - E - R - R - Y inside its <think block, each letter becomes its own distinct token in the context window . We proved this with three experiments: At every single token position t inside the cold-open <think trace, we appended a forced </think \nFinal Answer: suffix and measured the model's exact probability P "3" if it were forced to answer at that exact millisecond: What if the model just benefits from running extra forward passes “extra test-time compute” , regardless of what words it writes? We took 107 real thinking traces and replaced every thinking token inside <think with the exact same number of period dots . . . . . , giving the transformer the exact same number of forward passes and attention columns: Extra forward passes with dots do almost nothing 35.7% vs 33.6% . The semantic content written onto the scratchpad is the working memory. When Qwen3-4B finally closes </think and outputs "3", where do its attention heads look? Is the word "Wait" just random temperature noise? To find out, we measured the Shannon Entropy uncertainty across the vocabulary distribution, in bits at every token in the trace: H X = − Σᵢ pᵢ · log₂ pᵢ Across normal calculation tokens inside <think , the model is near-deterministic: the median entropy is just 0.012 bits . Right at discourse markers like "Wait" , "Hmm" , and "But" , however, token entropy spikes 30× to 200× higher—between 0.34 and 2.90 bits the 65th to 99.7th percentile of uncertainty . Recent interpretability research calls these high-entropy pivots "Forking Tokens" arXiv:2506.01939 https://arxiv.org/abs/2506.01939 : the sparse branching points where a reasoning trajectory can change direction. And when we ran our Logit Lens across all 36 layers at the exact moment "Wait" was generated, "Wait" did not become the 1 top candidate until Layer 33 of 36 . Self-doubt is computed in the very final layers of the transformer. We tested Qwen3-4B across multiple tasks and token budgets—and uncovered a nuanced reality that hype threads miss. On 100 GSM8K grade-school math problems 8 trials each, 800 completions : Notice that When the model is allowed to show its work out loud in the normal response, 90.6% vs 93.9% is within statistical noise. Similarly, on "strawberry", if you force a bare number with Thinking OFF, it gets 0 / 64 0% ; with Thinking ON, 256 / 256 100% ; and with Thinking OFF without the bare-number restriction, it spells the word out in the main reply and gets 20 / 20 100% . Where <think truly shines is on frontier competition problems where deep search and backtracking are required: on AIME 2024 Reported in the Qwen3 Technical Report, Tables 17–18 , Qwen3-4B jumps from 25.0% Thinking OFF to 73.8% Thinking ON . More thinking is not free — and on simple or deceptive problems, it can actively hurt: On our 100 GSM8K problems, Thinking Mode generated 1.84 million tokens compared to 0.24 million tokens with Thinking OFF — 7.7× more tokens and 6.6× higher median latency 25.9 seconds vs 3.9 seconds . Even though the user UI hides the <think block behind a collapsible dropdown, the GPU cannot forget a single thinking token while generating the response. Because every future token attends back to every past token, the GPU must store the Key K and Value V vectors for every token across every layer in high-bandwidth VRAM the KV-Cache . In Qwen3-4B bf16 = 2 bytes per parameter, 36 layers, 8 GQA key-value heads, head dim = 128 , the exact memory footprint per token is: KV bytes per token = 2 K,V × 36 layers × 8 KV heads × 128 dim × 2 bytes = 147,456 bytes ≈ 144 KB Why couldn’t we just prompt "Let's think step by step" like researchers did in 2022 Kojima et al., 2022; Wei et al., 2022 ? Because as Google's original Chain-of-Thought paper proved, zero-shot CoT "only yields performance gains when used with models of ~100B parameters" —smaller models produced fluent but illogical chains of thought. Today, a 4-billion-parameter model reasons effortlessly because of two training breakthroughs : Instead of paying humans to grade every intermediate line of math, DeepSeek-R1 arXiv:2501.12948 https://arxiv.org/abs/2501.12948 showed that you can give a model problems with automatically verifiable final answers math, code and train it via GRPO Shao et al., 2024 https://arxiv.org/abs/2402.03300 : Aᵢ = rᵢ − mean r₁, …, r G / std r₁, …, r G On a real GSM8K problem from our run where the true answer was 16, the model sampled 16, 4, 16, 16, 16, 4, 16, 4 → rewards 1, 0, 1, 1, 1, 0, 1, 0 mean = 0.625, std = 0.5175 . Every token inside the 5 correct traces got a positive nudge of +0.725 , while every token inside the 3 wrong traces got pushed down by -1.208 . To see this live, we trained Qwen3-0.6B-Base a tiny 600M-parameter base model for 250 steps of GRPO ~90 minutes on a single GPU : In DeepSeek-R1’s 671B-parameter RL run, reflective words like "Wait" spiked 5- to 7-fold during training. Yet in our tiny 0.6B GRPO run, "Wait" frequency actually dropped 0.004 → 0 as the small model locked onto a rigid "let's break it down step by step" template. Recent studies arXiv:2503.20783 https://arxiv.org/abs/2503.20783 show that RL amplifies self-correction behaviors that already exist in large base models 671B / 235B . And how did our Qwen3-4B model get so good at saying "Wait"? As documented in the Qwen3 Technical Report arXiv:2505.09388 https://arxiv.org/abs/2505.09388 , Qwen first trained its massive 235B and 32B teacher models with RL—and then trained the compact Qwen3-4B student via Strong-to-Weak Distillation , aligning its token-by-token logits directly with the teacher's probability distributions Why AI Says “Wait”: How Thinking Models Actually Work Every Step Measured https://pub.towardsai.net/why-ai-says-wait-how-thinking-models-actually-work-every-step-measured-b1baff6c592d was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.