How to count tokens across every major LLM provider in JavaScript

A friend of mine sent me a screenshot last month. His OpenAI bill for one day was $847. He runs a small SaaS product doing maybe $12,000 a month in revenue. The bill wasn't a bug, it wasn't a hack, and he hadn't shipped anything new for two weeks. What changed was that he was sending 1,800-token system prompts to gpt-4o on every classification call, and traffic had doubled.

Nobody had counted the tokens before sending them. Nobody had priced the call. So nobody noticed.

If you're building anything on top of an LLM API, the same trap is waiting for you. Here's how to count tokens properly across every major provider in JavaScript, and why the answer is more annoying than you'd expect.

OpenAI open-sourced their BPE tokenizer, so you can get exact counts client-side.

npm install gpt-tokenizer
js
import { encode } from 'gpt-tokenizer/encoding/cl100k_base'

const text = 'Your system prompt goes here.'
const tokens = encode(text).length
console.log(tokens) // 6

Two things to know:

cl100k_base

o200k_base

(different encoding, different token counts for the same text)Pick the encoding that matches the model you're calling. Mixing them silently gives you numbers that are close but not right.

import { encode as encodeO200k } from 'gpt-tokenizer/encoding/o200k_base'

const tokens = encodeO200k(text).length

Anthropic, Google, and DeepSeek do not publish a JavaScript tokenizer. There is no npm install claude-tokenizer

that gives you exact counts. This is annoying.

You have two options.

Option one: call the provider's own counting endpoint. Anthropic has /messages/count_tokens

. It returns the exact count but it's a network round-trip and costs latency. Fine for pre-flight in a batch job, awful in a hot path.

Option two: character-based estimation. Not perfect, but calibrated well enough for cost planning.

const CHARS_PER_TOKEN = {
  anthropic: 3.8,  // Claude BPE family
  google:    4.0,  // Gemini SentencePiece
  deepseek:  3.5,  // close to GPT
  llama:     3.8,  // Groq-hosted Llama variants
}

function estimateTokens(text, provider) {
  const cpt = CHARS_PER_TOKEN[provider] || 3.8
  return Math.max(1, Math.round(text.length / cpt))
}

estimateTokens('Your prompt', 'anthropic') // 3

These ratios come from published vendor guidance and community benchmarks on English text. On non-English text (Chinese, Japanese, Arabic), the ratios shift and estimates get less accurate. Flag anything you display as an estimate so you don't misrepresent it as exact.

One function that dispatches by model name:

import { encode as encodeCl100k } from 'gpt-tokenizer/encoding/cl100k_base'
import { encode as encodeO200k } from 'gpt-tokenizer/encoding/o200k_base'

function providerOf(model) {
  const m = model.toLowerCase()
  if (m.startsWith('gpt-') || m.startsWith('o1') || m.startsWith('o3')) return 'openai'
  if (m.startsWith('claude'))   return 'anthropic'
  if (m.startsWith('gemini'))   return 'google'
  if (m.startsWith('deepseek')) return 'deepseek'
  if (m.startsWith('llama'))    return 'llama'
  return 'unknown'
}

function openaiEncoding(model) {
  const m = model.toLowerCase()
  return (m.startsWith('gpt-4o') || m.startsWith('o1') || m.startsWith('o3'))
    ? 'o200k' : 'cl100k'
}

export function countTokens(text, model) {
  const provider = providerOf(model)

  if (provider === 'openai') {
    const enc = openaiEncoding(model) === 'o200k' ? encodeO200k : encodeCl100k
    return { count: enc(text).length, mode: 'exact' }
  }

  const cpt = { anthropic: 3.8, google: 4.0, deepseek: 3.5, llama: 3.8 }[provider] || 3.8
  return { count: Math.max(1, Math.round(text.length / cpt)), mode: 'estimate' }
}

Always surface the mode

to the caller. Estimates presented as exact numbers are the single biggest reason engineers lose trust in cost tools.

Once you have a count, the cost math is a lookup and a multiply.

const PRICE = {
  'gpt-4o':                   { in: 2.50, out: 10.00 },
  'gpt-4o-mini':              { in: 0.15, out: 0.60 },
  'claude-3-5-sonnet-latest': { in: 3.00, out: 15.00 },
  'gemini-1.5-pro':           { in: 1.25, out: 5.00 },
  'deepseek-chat':            { in: 0.14, out: 0.28 },
}

function costOf(inTokens, outTokens, model) {
  const p = PRICE[model]
  if (!p) return null
  return (inTokens / 1_000_000) * p.in + (outTokens / 1_000_000) * p.out
}

const inTok  = countTokens(prompt, 'claude-3-5-sonnet-latest').count
const outTok = 500
console.log(costOf(inTok, outTok, 'claude-3-5-sonnet-latest'))
// e.g. 0.0089 dollars, or roughly 89 cents per 100 calls

Rates are USD per one million tokens, from each provider's public pricing page. Keep this map updated because providers move prices more often than you'd expect.

If you don't want to wire all of this up yourself, I built a free calculator that does this exact thing across every model in one screen: tokensbill.com/tools/token-counter. Paste text, tick which models you care about, see the monthly cost side by side. Nothing leaves your browser.

But even if you use a hosted tool, wire the counting into your own code too. The moment you can see per-call cost in your own logs, you stop shipping features that are secretly ten times more expensive than the last one.

That's usually all it takes.