AI keeps whiffing on this simple question
There are no P’s in the word “Google,” but someone still needs to tell Gemini this — Google’s own AI Overview keeps suggesting there are P’s when asked.
Back in 2024, asking a model to count the R’s in “strawberry” became the internet’s canonical example of weird AI failure modes. Most models of that generation counted letters wrong about half of the time.
Since then, LLMs have made rapid leaps in everything from scientific reasoning to agentic coding, while major businesses have begun to rely on them in critical areas like finance. And yet, somehow, they still suffer on tasks like counting the R’s in “strawberry,” the P’s in Google, or the N’s in the days of the week, which I asked ChatGPT to do today:
How is it that models can now solve historic math problems but still fail to count letters in a word?
The typical explanation here has to do with “tokenization.” LLMs don’t read English characters; they break words into subwords, like “st -raw - berry.” Each segment of the word is a token, which is used because it’s the most efficient unit of language to compute (compared to the vast library of whole words). Under this theory, the counting issue has to do with how familiar an LLM is with a given token from its training data.
That explanation never quite made sense to me. “Strawberry” is a very common word. Even if it were tokenized several different ways within the training data (“st -raw - berry” and “straw - be -rry”), its tokens would still be quite familiar.
Keep reading with a 7-day free trial #
Subscribe to The Argument to keep reading this post and get 7 days of free access to the full post archives.