Three-plus years into the AI boom, we’re all still figuring out what an AI token is actually worth. Welcome to the era of stochastic shopping.
Tokens, at their core, are the tiny units of language that artificial intelligence systems ingest and process in order to reason, or at least mimic reasoning, and communicate with us. A token might be a punctuation mark, a word, or even part of a word. OpenAI suggests that, in English, one token is roughly equivalent to four characters, or about three-quarters of a word. By the same calculation, a paragraph might contain around 100 tokens, while roughly 1,500 words would amount to about 2,048 tokens.
This might sound scientific, but for people using consumer systems such as Claude or ChatGPT—rather than the application program interfaces used by developers and large companies—understanding what a token is and how much it is worth can feel surprisingly opaque. When you enter a prompt, you may receive an estimate of how many tokens the task consumed. What you do not receive is a clear explanation of why the task costs that particular number of tokens. The estimate can feel random and frustrating, especially when an AI provider suddenly tells you that you have hit your quota.
Users regularly complain about running up against those token limits. One Reddit user described hitting a cap and losing the emotional nuance they had developed in a chat. On X, some people post about hitting their limits quickly—as a matter of pride. But many have grown sensitive to perceived token caps and to the ways companies appear to throttle or restrict usage, particularly during periods of peak demand.
Of course, complicated tasks generally consume more tokens than simple ones. But token purchases can still feel nebulous in part because companies approach tokenization differently. Models also become personalized to users over time, potentially affecting how many tokens a system utilizes for a specific task.
There are also different types of tokens. Input tokens are used to process and interpret what a user submits. Output tokens correspond to the response the model generates. Cached tokens allow a system to reuse information it has previously processed rather than starting from scratch each time.
The models themselves can provide different token estimates in response to the same prompt. A friend and I recently experimented with this by asking the same OpenAI model to produce a simple timeline of American history. My response “cost” me twice as many tokens as his.
A study from researchers at the Stanford Digital Economy Lab published earlier this year found that AI models can vary widely in how many tokens they consume to complete the same task, sometimes by as much as 30 times. “This consumption-based pricing . . . has two major issues for consumers,” Jiaxin Pei, a postdoctoral researcher at Stanford who worked on the project, tells Fast Company. “You have no guarantee that this task will be done [well]. . . . Regardless . . . you have to pay the full price.”
The model token estimates are even more* *confusing because a task that might be expensive in terms of tokens for a model isn’t necessarily correlated with whether a task would be laborious for a human. Another wrinkle: That same Stanford research also notes that models underestimate their own token usage, which means it’s difficult to get a model to tell you, ahead of time, what you’re actually buying. (You can take a shot at guesstimating here, if you’d like).
All of this raises questions about the quickly fading token-maxxing moment. Yes, tokens buy access to something, but AI systems are fundamentally stochastic, or random, which means you don’t always know what you’re buying with tokens. Rather than functioning as discrete items with stable value, tokens are fuzzy units of consumption that only loosely correlate to the usefulness of the AI-generated response you get.
To be sure, people continue to embrace AI tools, even as they worry that AI companies have accumulated too much power. And tokens are hardly the only opaque currency in the modern economy—think billable hours, or airline points. Still, the confusion surrounding tokens reflects a broader sense that consumers are being asked to adopt AI before they can meaningfully understand its terms.
No wonder people are annoyed.