Every LLM provider publishes a pricing table.
$2.50 per million input tokens. $10 per million output tokens.` Clean. Transparent. Easy to spreadsheet.
So you run the napkin math: 10,000 requests/day × 2,000 input tokens × $2.50/M = $18.25/day on GPT-4o. Annualized: $6,660. The CFO approves it.
Three months later the bill is $54/day — $19,710/year — and nobody can explain the gap.
It's not a billing error. It's five structural leaks between the pricing page and your credit card.
| Leak | What It Is | How Much It Costs You |
|---|---|---|
| Workload ratio | Output tokens cost 3–4× more than input | 2.9× spread across use cases |
| Tokenizer variance | Same text = different token counts per provider | 5–15% (EN), 15–30% (multilingual) |
| Prompt caching | Anthropic gives 90% off, OpenAI 50% — nobody configures it | 24% of total bill |
| Batch processing | 50% off for async workloads | 15–30% blended |
| Retry overhead | Failed requests consume tokens twice | 1–3% + architectural waste |
These aren't additive. They're stackable. Combined, the difference between naive pricing and optimized reality is 40–65%.
Output tokens cost 3–5× more than input tokens. The ratio between them is determined by your workload — and it's the single largest cost variable.
Same model (GPT-4o). Same request count (10,000/day). Different workloads:
| Workload | Input | Output | Annual Cost | vs Chat |
|---|---|---|---|---|
| 💬 Chat | 20M | 8M | $47,450 | |
| 1× | ||||
| 🔍 RAG / Q&A | 60M | 8M | $83,950 | |
| 1.8× | ||||
| 📝 Summarization | 80M | 10M | $109,500 | |
| 2.3× | ||||
| 💻 Code Generation | 15M | 30M | $123,188 | |
| 2.6× | ||||
| 🌐 Translation | 30M | 30M | $136,875 | |
| 2.9× |
2.9× spread — same model, same request count. Before comparing providers. Before factoring any other leak.
Fix:Measure your actual input-to-output token ratio in production. Most teams guess 1:1. Almost no real workload is 1:1.
Every provider's tokenizer is different. The same text produces different token counts on each:
| Provider | Tokenizer | Relative Efficiency |
|---|---|---|
| OpenAI | ||
cl100k_base (tiktoken) |
||
| Baseline | ||
| Anthropic | Proprietary BPE | 5–10% fewer tokens (EN) |
| SentencePiece | 5–10% more tokens |
| DeepSeek | BPE (optimized for Chinese+English) | 5–15% more tokens (EN-only) | Why this matters: comparing per-token prices without benchmarking your actual text = comparing different units. Provider A at $2.00/M with a 10% hungrier tokenizer = Provider B at $2.20/M. The cheaper sticker price may be more expensive after tokenization.
Fix:Run your actual production text through 2–3 candidate tokenizers before committing. At 1M+ requests/day, a 10% efficiency gap is thousands/month.
Anthropic introduced prompt caching in August 2024. OpenAI followed with automatic caching. Google launched context caching in early 2025. The discounts are the largest cost lever in LLM APIs — and most teams never configure it.
| Provider | Standard Input | Cached Input | Discount |
|---|---|---|---|
| Anthropic Claude Opus 4 | $15.00/M | $1.50/M | 90% |
| Anthropic Claude Sonnet 4 | $3.00/M | $0.30/M | 90% |
| OpenAI GPT-4o | $2.50/M | $1.25/M | 50% |
| Google Gemini 2.5 Pro | $1.25/M | $0.3125/M | 75% |
What's actually cacheable in your app:
| Token Category | Typical Size | Cacheability |
|---|---|---|
| System prompt | 500–2,000 tokens | 100% |
| Few-shot examples | 500–3,000 tokens | 100% |
| RAG context | 2,000–8,000 tokens | 20–40% |
| Conversation history | 1,000–10,000 tokens | 0% |
Real example: a customer support chatbot with 1,500-token system prompt, 1,000-token few-shot examples, 3,000-token RAG context per query. Total input: 5,500 tokens. Cacheable: 2,500 tokens (45%).
| Scenario | Annual Cost (Claude Sonnet 4) |
|---|---|
| Naive (no caching) | $104,025 |
| With caching configured | $79,388 | Saved by one config change |
$24,638 (24%) | Fix:Identify your cacheable prefix tokens. Structure API calls so they appear at the beginning of every prompt. Anthropic requires explicit cache point marking; OpenAI and Google handle it automatically.
OpenAI and Anthropic offer batch endpoints at 50% off standard pricing. The tradeoff: up to 24-hour completion SLA instead of real-time response.
For offline workloads — evaluation runs, dataset labeling, embedding generation, nightly summarization, synthetic data generation — there is literally zero downside. The 50% discount is free money. Stacked with prompt caching:
plaintext
Cached input + batch = 5% of sticker price (90% off × 50% off)
Uncached input + batch = 50% of sticker price
Output + batch = 50% of sticker price
Moving 60% of the support chatbot's traffic to batch: $55,572/year vs $104,025 naive = 47% saved.
Fix:Segment traffic into realtime and async. Route async to batch endpoints. The infrastructure change is an API endpoint swap — no model changes, no prompt changes.
When your app hits API rate limits, the client retries — and the failed tokens are charged. At 2% retry rate, 10,000 requests/day: $365/year in wasted input tokens. Small, but the architectural cost is larger: teams over-provision multiple providers to avoid limits.
Fix:Exponential backoff with jitter. Monitor retry rate (if >1%, you need higher limits or a queuing layer). Route async traffic to batch endpoints (separate, higher limits).
| Tier | Models | Output Price | Best For |
|---|---|---|---|
| Premium | |||
| Claude Opus 4 | $75/M | Non-negotiable quality + caching | |
| Standard | |||
| GPT-4o, Claude Sonnet 4, Gemini 2.5 Pro, Mistral Large 2 | $5–15/M | General purpose | |
| Budget | |||
| GPT-4o-mini, Claude Haiku, Gemini Flash, Llama 4 Scout (Groq) | $0.50–1.25/M | Classification, extraction, filtering | |
| Disruptor | |||
| DeepSeek-V3, DeepSeek-R1 | $1.10–2.19/M | Flagship capability at budget prices |
The caching twist: Anthropic's 90% cache discount makes Claude Opus 4's effective cached input ($1.50/M) cheaper than GPT-4o's standard input ($2.50/M). At high cache hit rates, the premium tier beats the standard tier on price.
| Scale | GPU Cost | Breakeven vs DeepSeek | Breakeven vs GPT-4o-mini |
|---|---|---|---|
| 8B model | 1× H100 = $1,800/mo | Wins at 35% utilization | |
| Wins at 50% utilization | |||
| 70B model | 3× H100 = $5,400/mo | Wins at 40% utilization | |
| Wins at 3% utilization | |||
The utilization reality: most teams overestimate their GPU utilization. Self-hosted GPUs idle during nights, weekends, holidays. The API charges zero for idle time. Bursty traffic → API wins. Steady high throughput → self-hosting wins.
The hidden cost: self-hosting a 70B model across 3 GPUs requires understanding tensor parallelism, quantization (AWQ/GPTQ/FP8), continuous batching (vLLM/TGI), and GPU node management. Budget 0.25–0.5 FTE for production self-hosting.
Interactive calculator: jslet.com/llm-api-pricing-calculator — compare 12 models across 6 providers with caching, batch, and workload presets. All client-side, no signup.