Ask a modern AI to write a five-paragraph essay on the French Revolution, and it finishes in three seconds. Ask that same AI a question a seven-year-old can answer on their fingers:
“How many times does the letter *** r appear in*** *** strawberry?"***
With its “Thinking” switch turned off, the open model we tested (Qwen3-4B) answers instantly and confidently: "2."
Wrong. Then we flipped one switch, asked the exact same model again, and it answered: “3.”
Why does a machine that can write Python compilers and pass medical licensing exams fail to count three letters in a ten-letter word? And what actually happens inside those billions of parameters between the moment your finger hits Enter and the moment the first word appears on your screen?
ChatGPT’s proprietary weights are locked behind OpenAI’s API. But modern open-weight models like Qwen3–4B (released in 2025 under Apache 2.0) are built on the exact same transformer recipe. So we loaded Qwen3-4B and its pre-assistant base model (Qwen3-4B-Base) onto our own GPU, hooked probes into every single layer, and followed one simple question all the way through:
“Which city is the Eiffel Tower in?”
Every token ID, vector dimension, attention percentage, and probability below was measured directly from the live model.
🎥 Prefer watching the visual breakdown? Watch our full 13-minute animated explainer here:
The Misconception: “ChatGPT only reads my latest message, and it remembers who I am as we chat.”
When you press Enter, the neural network does not receive a neat little string containing only your question. In fact, the model has zero memory of you between messages.
Instead, the application wrapper constructs a single flat text document from scratch every single turn. It stitches together:
In Qwen3-4B, the raw text fed into the model looks like this:
<|im_start|>userWhich city is the Eiffel Tower in?<|im_end|><|im_start|>assistant
The model’s only job in the universe is to predict the next single piece of text that continues this document.
Wait — if a language model only predicts one piece of text, how does a full paragraph appear on your screen?
It runs in an autoregressive loop:
(Under the hood, production inference engines cache earlier layer calculations in GPU memory — the KV-Cache — so they don’t recompute past words from scratch, which we measure in Episode 2*).*
Takeaway #1: An LLM is a stateless next-piece predictor. The conversation history is the memory.
The Misconception: “AI reads words and letters the way humans do.”
Before your text ever touches a neural network layer, a deterministic preprocessor called a tokenizer chops your string into common chunks called tokens and swaps each chunk for an integer ID.
When we run "Which city is the Eiffel Tower in?" through OpenAI's GPT-4o-era tokenizer (o200k_base, vocabulary size 200,019), it splits into 8 tokens. Common short words stay whole, while longer words like "unbelievable" get cleaved into sub-word fragments: un | bel | ievable.
Now look at what happens to " strawberry":
import tiktoken
enc = tiktoken.get_encoding("o200k_base")print(enc.encode(" strawberry"))# Output: [101830]
In o200k_base, " strawberry" (with a leading space) is a single token: #101830 (and in Qwen3's tokenizer, a single token #72600).
The neural network never receives the characters s-t-r-a-w-b-e-r-r-y. It only sees the integer 101830. Asking an LLM to count the r's inside token #101830 without writing anything down is like asking you to count the letters in a word you have only ever heard spoken aloud as a barcode number.
Tokenization also explains why non-English languages cost more compute:
Because English dominated the tokenizer’s training corpus, English words get packed into single tokens while Indic scripts get shattered into smaller byte/character fragments — requiring nearly 2× the forward passes for the same sentence.
Takeaway #2: *LLMs read numbered chunks (*tokens), never individual letters.
An integer ID like 101830 has no mathematical meaning on its own—token #101830 isn't "closer" to token #101831 just because their IDs are adjacent.
So in Step 1 of the neural network, an Embedding Lookup Table swaps each token ID for a long list of floating-point numbers — a vector of length 2,560 in Qwen3-4B.
Think of those 2,560 numbers as GPS coordinates in a high-dimensional Map of Meaning. Nobody hand-coded this map; the coordinates were learned automatically during training so that words used in similar contexts land in the same neighborhood.
When we queried the real embedding table of Qwen3-4B for the nearest neighbors of specific tokens using cosine similarity, here is what lives right next door:
Even directions in this space carry approximate geometric meaning. If you take the vector step from v("France") → v("Paris") and apply that exact same offset starting from v("Japan"), the closest city vector you land on is "Tokyo".
(Where the popular analogy breaks down: this space has 2,560 dimensions, not 2, and vector arithmetic is noisy — when we tested v("India") + (v("Paris") - v("France")), the closest vector was still just "Delhi" because "Delhi" was already right next to "India").
Takeaway #3: Every token becomes a 2,560-number coordinate vector in a learned geometric map of meaning.
Static dictionary coordinates have a fatal flaw: context.
Consider this classic sentence from the 2017 Transformer paper:
“The animal didn’t cross the street because it was too tired.”
What does “it” refer to? You know immediately that it is the animal, not the street. How does the transformer figure that out?
Through Self-Attention. At each layer, every token broadcasts three learned vectors:
When we inspected Qwen3-4B processing that sentence, one attention head looking backward from the token "it" placed 92% of its attention weight directly on "animal" and 0.0% on "street".
Each of the 36 layers in Qwen3-4B runs 32 attention heads in parallel, and different heads specialize in different grammatical patterns without ever being explicitly programmed:
(One crucial rule: in autoregressive models like GPT and Qwen, attention is causal — words can only look backward at earlier tokens, never forward into the unwritten future).
Takeaway #4: Self-attention allows each token to dynamically route and blend information from relevant earlier tokens.
After the 32 attention heads mix context together, each token vector passes through a Feed-Forward Network (MLP / SwiGLU) — which researchers (Geva et al., 2021; Meng et al., 2022) have shown acts like a massive key-value memory bank storing factual associations.
One Attention block + one Feed-Forward block = 1 Transformer Layer. Qwen3-4B stacks 36 of these layers on top of each other.
Using an interpretability technique called the Logit Lens (projecting intermediate layer hidden states directly onto the output vocabulary matrix before the model reaches the top floor), we peeked at Qwen3-4B-Base's top guess at every floor as it processed "The Eiffel Tower is in the city of":
At the top of Layer 36, the neural network does not output a word. It outputs a list of ~151,936 raw scores (logits) — one for every token in its vocabulary — which are converted via Softmax into a probability distribution summing to 100%.
On "The Eiffel Tower is in the city of", Qwen3-4B-Base outputs:
To actually pick the next token, the sampler rolls a weighted die biased by a parameter called Temperature (****T):
That weighted dice roll is why clicking Regenerate gives you a different phrasing every time. The model’s weights never change while you chat; it is not “learning from you” in real time.
Where did those 4.02 billion numbers come from? Every modern chat LLM goes through three distinct training stages:
The model begins as random noise. It is fed trillions of tokens of internet text with one self-supervised objective: predict the next token. Every wrong guess triggers backpropagation, nudging billions of weights slightly toward the true token.
To put the scale in perspective: Meta’s open Llama 3 model was pre-trained on 15.6 trillion tokens across up to 16,000 H100 GPUs (3.8 × 10²⁵ FLOPs). If you read 8 hours a day at 250 words per minute, reading 15.6 trillion tokens would take you ~250,000 years.
A raw pre-trained model (Qwen3-4B-Base) is a document completer, not a helpful assistant—ask it a question and it might reply with three more worksheet questions. In SFT, human annotators write thousands of ideal <|im_start|>user ... <|im_start|>assistant conversations, teaching the model the turn-taking assistant format.
The model generates multiple candidate answers to a prompt, human raters rank them from best to worst, a Reward Model learns to predict human preferences, and the LLM is optimized to maximize that reward score. (Whenever ChatGPT shows you two side-by-side responses and asks “Which response do you prefer?”, you are generating RLHF preference data!).
The Misconception: “When AI hallucinates, it looked up the wrong row in its database.”
Remember: there is no database inside an LLM — only compressed statistical weights trained to produce plausible-sounding continuations.
To see hallucination happen live, we asked Qwen3-4B our exact same running question, but swapped "Eiffel" for a completely fictional name we invented:
“Which city is the Kesharpur Tower in?”
Without a millisecond of hesitation, Qwen3-4B replied:
“The Kesharpur Tower is located in Kesharpur, India… in the Bhagalpur district of Bihar… a symbol of the city.”
There is no Kesharpur Tower. Why did the model invent a detailed history in Bihar instead of saying “I don’t know”?
Three interlocking mechanisms cause this:
Finally, let’s return to our cold-open mystery. Why did flipping the Thinking switch allow the exact same Qwen3-4B model to get "strawberry" right (3 R's) when it previously failed (2 R's)?
Because with Thinking ON, before giving its final answer, the model wrote out a scratchpad:
<think>Let's spell out the word "strawberry" letter by letter:1: S2: T3: R (1)4: A5: W6: B7: E8: R (2)9: R (3)10: YThere are 3 R's.</think>
By spelling "s-t-r-a-w-b-e-r-r-y" separated by spaces and newlines, each individual letter became its own separate token on the page! Once the letters were broken out of token #101830 (#72600 in Qwen3) into individual tokens, the model's attention heads could actually see and count them.
Models need tokens to think.
Wait — what is that <think> switch under the hood? How did the model learn to write a scratchpad and suddenly say "Wait—" when it makes a mistake? And why does turning Thinking Mode ON sometimes make accuracy crash from 100% down to 37.5%?
That is the subject of Episode 2: 👉 Read Part 2: Why AI Says “Wait” — How Thinking Models Actually Work
Strawberry Has 2 R’s? How ChatGPT Actually Works Under the Hood (Every Number Measured) was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.