# Accurate local Claude 4.7+ tokenizer

> Source: <https://www.tokenize.rs/claude>
> Published: 2026-08-14 02:35:14+00:00

# Reconstructing Claude's tokenizer

[ctok on GitHub ↗](https://github.com/sanderland/ctok)
[Read the write-up ↗](/claude-tokenizer)

Claude publishes no tokenizer. `count_tokens`

hands back a
number, one round trip at a time, and nothing about where it came
from. ctok returns the same number offline, and the pieces behind it.

Our best estimate of the vocabularies: 49,152 entries in v3, a third of that in v4.7 at 16,384, with some of both reserved for image tokens.

Markers ride inside the piece they belong to: word edges, capitals,
and raw bytes where the vocabulary has no piece for a character.
[The write-up](/claude-tokenizer) goes through them.

## The cost of a small vocabulary

Claude 5's tokenizer comes at a cost, but less than its size suggests. Its vocabulary is about 12× smaller than o200k, while its token bill is 1.24× to 2.44× larger on these samples. Common pieces do most of the compression; a larger vocabulary spends the rest on rarer words and scripts.

Claude Opus 5 and GPT-5.6 Sol both charge $5 per million input tokens, so the token ratio is the cost ratio.