This article was written with the assistance of AI, under human supervision and review.
Most Claude Code cost overruns stem from invisible context accumulation and cache misses that the billing dashboard never surfaces. Production teams ship AI-powered features, watch token spend double month-over-month, and trace the issue to conversation histories that ballooned from 10k to 200k tokens without a single code change. The billing line items show "input tokens" and "cached tokens," but they omit the cascading cost when a cache invalidates mid-session or when preprocessing hooks fire redundant model calls. The result is a budget crisis that looks like normal usage until the invoice arrives.
%% alt: Problem flow showing silent context growth leading to cost explosion
The corrective pattern is straightforward: set hard token budgets per request, implement prompt caching with explicit TTL tracking, and build a cost-aware context manager that truncates or summarizes before thresholds break. This approach prevents runaway costs at the API boundary rather than reacting to billing alerts after the damage compounds.
%% alt: Solution flow showing budget enforcement preventing cost overruns
This post covers token budget implementation, prompt caching mechanics that actually reduce costs in multi-turn sessions, the cumulative context patterns the dashboard hides, and production architectures that enforce spend limits without breaking agent workflows.
Token budgets act as circuit breakers that prevent a single request from consuming excessive API credits. Most Claude Code cost explosions originate from workflows that accumulate context across multi-turn conversations — each exchange appends messages, tool results, and thinking tokens to the session history, and without a ceiling, the input token count climbs exponentially.
The distinction between soft and hard budgets is critical. A soft budget logs a warning when token usage exceeds a threshold but allows the request to proceed. A hard budget rejects the call or truncates the context before sending it. Production systems require hard budgets because warnings accumulate into budget overruns — a developer ignores five "high token usage" alerts, and the month-end invoice reflects 50 calls that each burned 100k tokens at full rate.
%% alt: Token budget enforcement hierarchy showing soft vs hard limits
The implementation pattern centers on calculating token counts before the API boundary. Claude Code's SDK does not expose a built-in tokenizer, so production systems either estimate tokens using byte-length heuristics (1 token ≈ 4 characters for English text) or call a lightweight tokenizer library. The tradeoff is accuracy — heuristics undercount for code-heavy context, tokenizers add latency — but both approaches beat unbounded spend.
A hard budget implementation throws an error or truncates the oldest messages when the total exceeds the limit. Truncation preserves recent context while discarding history, which maintains agent continuity at the cost of losing earlier conversation threads. The alternative — summarization — compresses old messages into a condensed prompt, but that adds a preprocessing step that itself consumes tokens. For cost-sensitive workflows, truncation is cheaper.
A production-grade token budget guard wraps the Claude API client with a pre-call check that estimates or measures token usage. The guard enforces a per-request ceiling and a per-session cumulative limit, so individual calls stay within bounds and multi-turn conversations do not drift into uncapped territory.
The following implementation uses a simple character-based heuristic for token estimation and truncates the message array when limits breach:
interface TokenBudgetConfig {
maxTokensPerRequest: number;
maxTokensPerSession: number;
estimateRatio: number; // characters per token, default 4
}
class TokenBudgetGuard {
private sessionTokens = 0;
constructor(private config: TokenBudgetConfig) {}
estimateTokens(text: string): number {
return Math.ceil(text.length / this.config.estimateRatio);
}
enforceRequestBudget(messages: Array<{ role: string; content: string }>): Array<{ role: string; content: string }> {
let totalTokens = 0;
const estimatedMessages = messages.map(msg => ({
...msg,
estimatedTokens: this.estimateTokens(msg.content)
}));
totalTokens = estimatedMessages.reduce((sum, msg) => sum + msg.estimatedTokens, 0);
if (totalTokens > this.config.maxTokensPerRequest) {
// Truncate oldest messages until under budget
const truncated = [...estimatedMessages];
while (totalTokens > this.config.maxTokensPerRequest && truncated.length > 1) {
const removed = truncated.shift()!;
totalTokens -= removed.estimatedTokens;
}
console.warn(`Token budget exceeded, truncated ${estimatedMessages.length - truncated.length} messages`);
return truncated.map(({ estimatedTokens, ...msg }) => msg);
}
return messages;
}
enforceSessionBudget(requestTokens: number): void {
this.sessionTokens += requestTokens;
if (this.sessionTokens > this.config.maxTokensPerSession) {
throw new Error(
`Session token budget exhausted: ${this.sessionTokens}/${this.config.maxTokensPerSession}`
);
}
}
resetSession(): void {
this.sessionTokens = 0;
}
}
// Usage in a Claude Code workflow
const budgetGuard = new TokenBudgetGuard({
maxTokensPerRequest: 50000,
maxTokensPerSession: 200000,
estimateRatio: 4
});
async function sendClaudeRequest(messages: Array<{ role: string; content: string }>) {
const truncatedMessages = budgetGuard.enforceRequestBudget(messages);
const requestTokens = truncatedMessages.reduce(
(sum, msg) => sum + budgetGuard.estimateTokens(msg.content),
0
);
budgetGuard.enforceSessionBudget(requestTokens);
// Proceed with API call using truncatedMessages
// const response = await claudeClient.messages.create({ messages: truncatedMessages, ... });
}
This pattern enforces both per-request and cumulative session limits. The enforceRequestBudget
method truncates from the oldest messages first, preserving recent context. The enforceSessionBudget
method throws when the session total exceeds the ceiling, forcing the caller to reset or terminate the conversation. Production systems extend this with actual tokenizer libraries like js-tiktoken
for GPT-style tokenization or Anthropic's upcoming tokenizer API when available.
The failure mode here is subtle but expensive: if the heuristic underestimates tokens, the API call proceeds with more tokens than budgeted, and costs accumulate silently. The safeguard is to set conservative estimates (3 characters per token instead of 4) and log discrepancies when actual billing data reveals overcounts.
Prompt caching reduces costs by reusing previously processed context across API calls. Claude Code charges lower rates for cached input tokens — as of 2026, cached tokens cost roughly 10% of cold-read input tokens. The implication here is straightforward: a cache hit on 50k tokens saves 90% of the input token cost, but cache invalidations force cold reads that erase those savings.
The caching mechanism is prefix-based. Claude caches the longest common prefix of the messages array, so if Call A sends [system, user1, assistant1]
and Call B sends [system, user1, assistant1, user2]
, the first three messages hit the cache and only user2
reads cold. The cache persists for 5 minutes by default, so a multi-turn conversation that completes within that window maximizes hits.
%% alt: Prompt caching flow showing cache hit vs cold read cost paths
The failure mode occurs when cache invalidations cascade across sessions. If a system prompt changes mid-conversation, the entire prefix invalidates, and every subsequent call reads cold. Similarly, if the message order shifts — for example, a preprocessing hook reorders tool results — the cache misses. The cost delta is severe: a 10-call session with consistent caching costs 10% of input tokens after the first call, but a session with 10 cold reads costs 10x.
Production caching strategies enforce these rules:
The distinction between development and production caching is critical. Development workflows often mutate prompts for iteration, so cache hits are rare and costs stay low due to small message volumes. Production workflows with stable prompts and high call frequency see dramatic savings from caching, but only if the architecture respects prefix stability.
A cost-aware context manager wraps the conversation history with logic that tracks token usage, enforces caching rules, and compresses or truncates context when budgets approach limits. The manager acts as the single source of truth for session state, preventing ad-hoc message array mutations that break caching or exceed budgets.
The core responsibilities are:
Here is a TypeScript implementation:
interface Message {
role: 'system' | 'user' | 'assistant';
content: string;
}
interface ContextManagerConfig {
maxTokensPerSession: number;
compressionThreshold: number; // trigger compression at this token count
estimateRatio: number;
}
class CostAwareContextManager {
private messages: Message[] = [];
private totalTokens = 0;
constructor(private config: ContextManagerConfig) {}
private estimateTokens(text: string): number {
return Math.ceil(text.length / this.config.estimateRatio);
}
addMessage(message: Message): void {
const tokens = this.estimateTokens(message.content);
if (this.totalTokens + tokens > this.config.maxTokensPerSession) {
throw new Error(
`Adding message would exceed session budget: ${this.totalTokens + tokens}/${this.config.maxTokensPerSession}`
);
}
this.messages.push(message);
this.totalTokens += tokens;
if (this.totalTokens >= this.config.compressionThreshold) {
this.compress();
}
}
private compress(): void {
// Summarize older messages to reduce token count
// This example truncates, but production systems use an LLM call to summarize
const keepRecent = 3; // keep last 3 messages for continuity
const toCompress = this.messages.slice(0, -keepRecent);
if (toCompress.length === 0) return;
const summary = `[Summarized ${toCompress.length} earlier messages: conversation history compressed to preserve context within token budget]`;
const summaryTokens = this.estimateTokens(summary);
this.messages = [
{ role: 'system', content: summary },
...this.messages.slice(-keepRecent)
];
this.totalTokens = summaryTokens + this.messages.slice(1).reduce(
(sum, msg) => sum + this.estimateTokens(msg.content),
0
);
console.log(`Context compressed: ${toCompress.length} messages summarized, ${this.totalTokens} tokens remaining`);
}
getMessages(): Message[] {
return [...this.messages]; // return copy to prevent external mutation
}
getTotalTokens(): number {
return this.totalTokens;
}
reset(): void {
this.messages = [];
this.totalTokens = 0;
}
}
// Usage
const contextManager = new CostAwareContextManager({
maxTokensPerSession: 150000,
compressionThreshold: 100000,
estimateRatio: 4
});
contextManager.addMessage({ role: 'system', content: 'You are a helpful assistant.' });
contextManager.addMessage({ role: 'user', content: 'Explain dependency injection.' });
// ... conversation continues
// Compression triggers automatically at 100k tokens
const messages = contextManager.getMessages();
// Use messages in Claude API call
This implementation compresses context by summarizing older messages when the token count exceeds the threshold. The summarization here is trivial — production systems call an LLM with a "summarize this conversation" prompt, which itself consumes tokens but reduces the cumulative count. The tradeoff is accuracy: aggressive compression loses detail, but it prevents session termination due to budget exhaustion.
The failure mode here is premature compression. If the threshold is too low, the manager compresses after every few turns, and the conversation loses coherence. If too high, compression triggers too late, and the next message addition exceeds the budget. Calibration depends on the workflow — customer support sessions with long histories benefit from aggressive compression, while code generation workflows with short exchanges tolerate higher thresholds.
The Anthropic billing dashboard aggregates token usage into high-level categories: input tokens, output tokens, cached input tokens. What it omits is the per-session breakdown that reveals cost patterns invisible in monthly totals. A team sees "2 million cached tokens" and assumes caching is working, but the dashboard does not show that 80% of those tokens came from cache misses due to mid-session prompt mutations.
The hidden cost patterns are:
%% alt: Hidden cost patterns showing cumulative drift and cache invalidation
The corrective pattern is instrumentation at the session level. Production systems log token counts, cache hit rates, and model selection per call, then aggregate this data into a cost dashboard that surfaces the patterns the billing API hides. The implementation is straightforward: wrap the Claude client with a logging layer that records metadata before and after each call.
For example, track these metrics per session:
Aggregate these metrics weekly and compare to the billing dashboard totals. Discrepancies reveal hidden costs — if the dashboard shows 1 million input tokens but session logs show 2 million due to preprocessing overhead, the team knows to optimize hooks before the next invoice.
A production cost control architecture combines preprocessing hooks, model selection gates, and budget enforcement at multiple layers. The goal is to prevent expensive API calls before they occur, rather than reacting to costs after the fact.
The architecture operates in three stages:
%% alt: Production cost control flow showing preprocessing, routing, and enforcement stages
Preprocessing hooks are the first cost control point. A hook that detects duplicate messages in the context and removes them improves cache hit rates without losing information. A hook that truncates code snippets longer than 10k characters prevents token bloat from large file diffs. The failure mode is over-aggressive preprocessing — a hook that removes too much context breaks agent workflows. Calibration requires A/B testing hooks against accuracy benchmarks.
Model selection gates operate on request metadata. If the user query is under 50 tokens and does not mention "reasoning" or "explain," route to Claude Code Standard. If the query requests multi-step planning or code generation with dependencies, route to Extended Thinking. The tradeoff is latency — Extended Thinking adds seconds to response time but provides higher accuracy for complex tasks. The cost delta is significant: Extended Thinking costs 2-3x per token compared to Standard.
Budget enforcement happens at the API client layer. The guard from the earlier section checks per-request limits and throws before the call. A separate session-level guard tracks cumulative spend and terminates the session when monthly budgets approach exhaustion. The guard logs rejection events for debugging — if users report broken workflows, the logs reveal whether budget enforcement was the cause.
This matters because cost control without observability creates silent failures. A budget guard that rejects requests but does not log the event leaves teams debugging "random errors" without realizing the root cause is token limits.
Cached input tokens cost approximately 10% of cold-read tokens, so a 50k token cache hit saves 90% of input costs compared to a cold read. In a 10-call session, caching after the first call reduces total input costs to roughly 20% of the uncached equivalent, assuming the cache stays valid.
Any mutation to the message array prefix invalidates the cache, forcing all subsequent calls to read cold. If a preprocessing hook reorders messages or edits prior content, the cache misses, and costs revert to full cold-read rates. The corrective pattern is to apply hooks before the first call or ensure hooks append new messages rather than mutating existing ones.
Hard budget guards that truncate context can break workflows if critical information gets removed. The safeguard is to set budgets high enough to accommodate the longest expected conversation and implement compression (summarization) instead of truncation, so older context condenses rather than disappears. The tradeoff is accuracy loss from summarization versus unbounded spend from uncapped context.
Extended Thinking charges 2-3x per input token compared to Standard and adds "thinking tokens" that count toward total usage. A 10k token request to Extended Thinking costs 20-30k token-equivalents due to internal reasoning steps. The cost justifies itself for complex multi-step tasks but inflates budgets for simple queries.
Track per-session token growth (start vs end counts), cache hit rates (cached tokens / total input tokens), model selection distribution (Standard vs Extended Thinking call ratios), and preprocessing hook overhead (tokens added by hooks before API calls). Aggregate these weekly and compare to billing dashboard totals to surface discrepancies.
Production cost control requires monitoring that surfaces spend patterns before monthly invoices reveal overruns. The corrective pattern is to instrument the API client layer with per-call metadata logging and aggregate this data into cost dashboards that track token usage, cache efficiency, and budget burn rates in real time.
The essential metrics are:
Set alerts at 75% and 90% of monthly budget thresholds. A 75% alert gives teams a week to optimize before exhaustion; a 90% alert triggers immediate action — freeze non-critical workflows, enforce aggressive truncation, or switch to cheaper models.
The failure mode is alert fatigue. If thresholds are too low, teams ignore alerts, and budgets exhaust anyway. If thresholds are too high, alerts fire too late to prevent overruns. Calibration requires historical data — analyze past months to determine typical burn rates and set thresholds that fire with 5-7 days of budget runway remaining.
That covers the essential patterns for Claude Code cost control in production. Implement token budgets at the request level, enforce prompt caching with stable prefixes, build cost-aware context managers that compress before limits break, and monitor spend in real time so alerts fire before invoices arrive. Apply these in production and the difference will be immediate.