{"slug": "tokenspeed", "title": "Tokenspeed", "summary": "A new interactive tool called Tokenspeed lets users visualize and compare LLM token-generation rates across four modes—code, text, think, and agent—by simulating speeds from 5 to 800 tok/s. It also demonstrates prefill latency, showing how long-context prompts delay first-token output, a metric often omitted from benchmark throughput numbers.", "body_md": "Every local-LLM benchmark reports throughput: *\"47 tok/s on an M3,\"*\n*\"180 tok/s on a 4090,\"* *\"500 tok/s on Groq.\"*\nUnless you've actually watched tokens stream at those rates, the numbers are\nhard to internalize. This is the rendering.\n\n### Four modes\n\n**code**— syntax-highlighted pseudo-code, the most common thing you watch stream out of an LLM.** text**— lorem ipsum prose, for the chat/answer case.** think**— dim-italic reasoning sentences alternating with code, mimicking a reasoning model thinking out loud.** agent**— alternating tool calls and code generation with processing pauses, simulating an AI coding agent.\n\n### What to try\n\nStart at the default `30` and read along. Then hit\n`1` (5 tok/s — Raspberry-Pi-class local model),\n`5` (60 tok/s — typical hosted Claude or GPT),\n`7` (200 tok/s — Groq territory),\n`9` (800 tok/s — Cerebras-class, where the bottleneck is your eyeballs).\n\nNow switch between `c` and `t` at the same rate.\nThe difference is striking — and intentional.\n\n### Prompt processing\n\nBefore a model emits a single output token, it has to read your whole\nprompt — the *prefill* pass. Open `p`, pick a context size,\nand the tool makes you sit through that wait before streaming starts, the\nsame way it makes you feel tok/s. Prefill is much faster per token than\ngeneration, but a long context still stalls you: 64k tokens at 1,000 tok/s\nof prefill is over a minute of nothing. That delay is time-to-first-token,\nand it's the half of the latency story a throughput number never shows.\n\n### What counts as a token\n\nThis approximates BPE-style tokenization, not any vendor-specific encoder\n(`tiktoken`\n\n, Claude's tokenizer, etc. — those disagree in the\ndetails anyway).\n\nShort words are often one token; longer identifiers split into chunks\n(`processUserInput`\n\n→ `process`\n\n+ `User`\n\n+ `Input`\n\n);\npunctuation and operators usually count too.\n\nCode is more token-dense than prose, so the same tok/s can feel very different depending on what's streaming. The benchmark number is honest; the perceptual effect varies a lot by content type — which is the gap this tool exists to expose.\n\nEnglish prose averages ~1.3 tokens per word, so 30 tok/s ≈ 23 words/s.", "url": "https://wpnews.pro/news/tokenspeed", "canonical_source": "https://mikeveerman.github.io/tokenspeed/?rate=30&mode=code", "published_at": "2026-07-15 21:18:02+00:00", "updated_at": "2026-07-15 21:28:10.379834+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-infrastructure"], "entities": ["Groq", "Cerebras", "Claude", "GPT", "Raspberry Pi"], "alternates": {"html": "https://wpnews.pro/news/tokenspeed", "markdown": "https://wpnews.pro/news/tokenspeed.md", "text": "https://wpnews.pro/news/tokenspeed.txt", "jsonld": "https://wpnews.pro/news/tokenspeed.jsonld"}}