22:22
2026-10-06
actual.inc
ai-infrastructure
toks: the best tokenizer on Earth
Actual Computer released toks 0.3.0, a tokenizer it says runs 13x to 151x faster than Hugging Face tokenizers and beats tiktoken in all 195 cells it can run, while returning identical token ids to Hug…