03:39
2026-08-25
ianbarber.blog
large-language-models
Vocab Break
Tokenizer enthusiast Sander Land reproduced a tokenizer similar to Claude's current one and found it has only about 15,000 entries, far fewer than Qwen 3.8's 250,000 tokens. Anthropic's tokenizer size…