Member-only story
Most developers still treat a token as “roughly a word.” That shortcut was fine in 2023. It breaks down now.
In July 2026, Anthropic alone runs five active model tiers** (Haiku 4.5, Sonnet 5, Opus 4.8, Fable 5, Mythos 5),** OpenAI just shipped a three-tier GPT-5.6 family **(Sol, Terra, Luna), **and Google has Gemini 3.1 Pro and 3.5 Flash in production.
Each one tokenizes text differently, prices it differently, and degrades differently as context grows.
Claude’s own newer tokenizer, for example, produces roughly 30 percent more tokens for identical text than the tokenizer used by Sonnet 4.6 and earlier models. “About four characters per token” is an approximation that quietly stops being true depending on which model you’re calling.
This piece skips the hand-wavy version. Every claim below is either something you can run yourself with no API key, or a number pulled from a provider’s own documentation, dated and sourced, because these figures move fast enough that “current” has a shelf life of weeks. You’ll leave with a runnable tokenizer, a…