cd/entity/bpe-openai· home› entities› bpe-openai
grep -l @bpe-openai /news/*.json | wc -l → 1

bpe-openai

mentions 1 type Organization feed RSS

// recent coverage 1 mentions

02:14
2026-06-16
discuss.huggingface.co
developer-tools

Quicktok: A faster tokenizer

Developer dmatth1 released quicktok, a fast exact BPE tokenizer written in C++ that runs 2–3.5× faster than bpe-openai and 4–11× faster than tiktoken on CPU. The tokenizer is byte-identical to tiktoke…

// co-occurs with top 7 entities
dmatth1 1 quicktok 1 tiktoken 1 OpenAI 1 llama.cpp 1 GitHub 1 Apple M1 1