How to count tokens across every major LLM provider in JavaScript

A developer has published a guide to counting tokens across major LLM providers in JavaScript, highlighting that OpenAI's tokenizer is open-source while Anthropic, Google, and DeepSeek lack JavaScript tokenizers. The guide provides code for exact OpenAI counts and character-based estimation for other providers, emphasizing the importance of accurate token counting to avoid unexpected API costs.

A friend of mine sent me a screenshot last month. His OpenAI bill for one day was $847. He runs a small SaaS product doing maybe $12,000 a month in revenue. The bill wasn't a bug, it wasn't a hack, and he hadn't shipped anything new for two weeks. What changed was that he was sending 1,800-token system prompts to gpt-4o on every classification call, and traffic had doubled. Nobody had counted the tokens before sending them. Nobody had priced the call. So nobody noticed. If you're building anything on top of an LLM API, the same trap is waiting for you. Here's how to count tokens properly across every major provider in JavaScript, and why the answer is more annoying than you'd expect. OpenAI open-sourced their BPE tokenizer, so you can get exact counts client-side. npm install gpt-tokenizer js import { encode } from 'gpt-tokenizer/encoding/cl100k base' const text = 'Your system prompt goes here.' const tokens = encode text .length console.log tokens // 6 Two things to know: cl100k base o200k base different encoding, different token counts for the same text Pick the encoding that matches the model you're calling. Mixing them silently gives you numbers that are close but not right. js import { encode as encodeO200k } from 'gpt-tokenizer/encoding/o200k base' const tokens = encodeO200k text .length Anthropic, Google, and DeepSeek do not publish a JavaScript tokenizer. There is no npm install claude-tokenizer that gives you exact counts. This is annoying. You have two options. Option one : call the provider's own counting endpoint. Anthropic has /messages/count tokens . It returns the exact count but it's a network round-trip and costs latency. Fine for pre-flight in a batch job, awful in a hot path. Option two : character-based estimation. Not perfect, but calibrated well enough for cost planning. js const CHARS PER TOKEN = { anthropic: 3.8, // Claude BPE family google: 4.0, // Gemini SentencePiece deepseek: 3.5, // close to GPT llama: 3.8, // Groq-hosted Llama variants } function estimateTokens text, provider { const cpt = CHARS PER TOKEN provider || 3.8 return Math.max 1, Math.round text.length / cpt } estimateTokens 'Your prompt', 'anthropic' // 3 These ratios come from published vendor guidance and community benchmarks on English text. On non-English text Chinese, Japanese, Arabic , the ratios shift and estimates get less accurate. Flag anything you display as an estimate so you don't misrepresent it as exact. One function that dispatches by model name: js import { encode as encodeCl100k } from 'gpt-tokenizer/encoding/cl100k base' import { encode as encodeO200k } from 'gpt-tokenizer/encoding/o200k base' function providerOf model { const m = model.toLowerCase if m.startsWith 'gpt-' || m.startsWith 'o1' || m.startsWith 'o3' return 'openai' if m.startsWith 'claude' return 'anthropic' if m.startsWith 'gemini' return 'google' if m.startsWith 'deepseek' return 'deepseek' if m.startsWith 'llama' return 'llama' return 'unknown' } function openaiEncoding model { const m = model.toLowerCase return m.startsWith 'gpt-4o' || m.startsWith 'o1' || m.startsWith 'o3' ? 'o200k' : 'cl100k' } export function countTokens text, model { const provider = providerOf model if provider === 'openai' { const enc = openaiEncoding model === 'o200k' ? encodeO200k : encodeCl100k return { count: enc text .length, mode: 'exact' } } const cpt = { anthropic: 3.8, google: 4.0, deepseek: 3.5, llama: 3.8 } provider || 3.8 return { count: Math.max 1, Math.round text.length / cpt , mode: 'estimate' } } Always surface the mode to the caller. Estimates presented as exact numbers are the single biggest reason engineers lose trust in cost tools. Once you have a count, the cost math is a lookup and a multiply. js const PRICE = { 'gpt-4o': { in: 2.50, out: 10.00 }, 'gpt-4o-mini': { in: 0.15, out: 0.60 }, 'claude-3-5-sonnet-latest': { in: 3.00, out: 15.00 }, 'gemini-1.5-pro': { in: 1.25, out: 5.00 }, 'deepseek-chat': { in: 0.14, out: 0.28 }, } function costOf inTokens, outTokens, model { const p = PRICE model if p return null return inTokens / 1 000 000 p.in + outTokens / 1 000 000 p.out } const inTok = countTokens prompt, 'claude-3-5-sonnet-latest' .count const outTok = 500 console.log costOf inTok, outTok, 'claude-3-5-sonnet-latest' // e.g. 0.0089 dollars, or roughly 89 cents per 100 calls Rates are USD per one million tokens, from each provider's public pricing page. Keep this map updated because providers move prices more often than you'd expect. If you don't want to wire all of this up yourself, I built a free calculator that does this exact thing across every model in one screen: tokensbill.com/tools/token-counter https://tokensbill.com/tools/token-counter . Paste text, tick which models you care about, see the monthly cost side by side. Nothing leaves your browser. But even if you use a hosted tool, wire the counting into your own code too. The moment you can see per-call cost in your own logs, you stop shipping features that are secretly ten times more expensive than the last one. That's usually all it takes.