{"slug": "count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you", "title": "Count tokens before you ship so agent bills do not surprise you", "summary": "A developer published a practical guide arguing that teams should measure token consumption before shipping agent features, since context windows, per-token pricing and latency all hinge on the same unit. The writeup notes that chat history, tool definitions and tool results all consume the context budget and that agents multiply costs across planning, tool calls and retries, and it offers a sketch cost function that sums input and output tokens across every call. It cites Anthropic's price list, where output costs five times input, and Cisco president Jeetu Patel's argument that \"price per token could turn into a vanity metric\" while the price of the outcome is what matters.", "body_md": "Your API dashboard shows a spike, but the feature \"felt small.\" Look at the tokens: what the model reads, writes, and re-sends on every step. [Tokenization](https://chainofthought.show/glossary/tokenization/) is not academic. Context limits, per-token pricing, and latency all use the same unit, and dense code or odd formatting can cost more than the word count suggests.\n\nLanguage models do not read whole words or raw characters. They read tokens, sub-word chunks from a fixed vocabulary. In English, a common word is often one token; rare words and code can split into several, and punctuation and spaces count too. OpenAI's rule of thumb is about four characters per token, or about three-quarters of a word, so 100 tokens is roughly 75 words.\n\nThat ratio drifts. Non-English text, code, and weird formatting usually burn more tokens per visible line. Before you argue about model tier, paste representative prompts into a tokenizer and note input tokens per request. Guessing from word count is how teams under-budget chat history and tool output.\n\nThe [context window](https://chainofthought.show/glossary/context-window/) is a token budget for one response, and the latest user message is only one part of it. System instructions, prior turns, tool definitions, and tool results all sit in that budget. Output tokens need room too. In chat, unless your app trims or summarizes, the full history goes back on the next turn, so input tokens climb every step.\n\nExceeding the limit is not one universal behavior. Depending on the API, the request may be rejected or generation may stop, and many applications truncate or compact the input before sending it. Compaction changes what the model sees next; it does not make the window bigger. Anthropic's documentation also warns that accuracy and recall can degrade as context grows, what it calls context rot. For cost planning, whether a prompt fits in the window and whether it is cheap to keep sending are different questions.\n\nFor most teams on hosted models, [inference](https://chainofthought.show/glossary/inference/) is the bill that repeats: every message and agent step is another run. Training is someone else's capex; your product pays per call.\n\nProviders charge input and output separately. On Anthropic's price list as of October 2026, output costs five times as much as input for each Claude model. Reasoning models can bill hidden reasoning as output, so a short visible answer can still be expensive. Repeated static instructions may qualify for prompt caching: Anthropic charges about a tenth of normal input to read a cached block, with a higher cost the first time you write it.\n\nOne successful task is rarely one call. Agents plan, call tools, retry, and often carry a growing context each time. Cost per task is roughly the sum over every call of (input tokens × input price) plus (output tokens × output price), including failures and retries.\n\nJeetu Patel, Cisco's president and chief product officer, argued in episode 67 that price per token could turn into a vanity metric: the price per token falls while long-running agents use more tokens. What matters to him is the price of the outcome. Your estimator should still start with tokens, because that is what you control in design.\n\nUse your provider's current input and output rates. If you only remember the Anthropic ratio above, set `OUTPUT_PRICE = 5 * INPUT_PRICE` as a placeholder until you paste real numbers from the price list.\n\nLabel this a sketch, not production accounting:\n\n``` bash\n# Sketch: monthly inference cost from volume.\n# Prices are $ per 1M tokens; replace the placeholders with your provider's list.\ndef monthly_cost(tasks_per_month, calls_per_task, avg_input_tokens,\n                 avg_output_tokens, input_price, output_price):\n    calls = tasks_per_month * calls_per_task\n    per_call = (avg_input_tokens * input_price\n                + avg_output_tokens * output_price) / 1_000_000\n    return calls * per_call\n\nINPUT_PRICE = 1.0                 # placeholder\nOUTPUT_PRICE = 5 * INPUT_PRICE    # the ratio on Anthropic's list\n# 50k tasks, 8 model calls each, 12k tokens in and 800 out per call\nbill = monthly_cost(50_000, 8, 12_000, 800, INPUT_PRICE, OUTPUT_PRICE)\n```\n\nRun three scenarios on the same workflow: today's averages, a heavy month (longer history, more retries), and a trimmed prompt (fewer input tokens or fewer calls per task). The line that moves the total most is where you spend engineering time.\n\nSet `calls_per_task` realistically for agents. A single user action might be dozens of inferences. Divide total cost by completed tasks, not by API calls, when you compare features.\n\nI think that today a lot of organizations are not thinking about their token spend regularly and they're getting hit with their bills.\n\n*Jiaona Zhang, Laurel*\n\n[Jiaona Zhang](https://chainofthought.show/guests/jiaona-zhang/) ties that to \"token maxxing\": mandating AI use without asking whether the task earned the tokens, like using a model to redo slide fonts when one click would do. The estimator is how you catch that before finance does.\n\nIf output tokens dominate (long answers, reasoning traces, verbose tool JSON), tighten schemas, cap completion length, or route easy steps to a smaller model. If input tokens dominate (system prompt, full chat log, fat tool results), trim history, summarize older turns, cache stable instructions, and stop pasting whole documents when a chunk will do.\n\nJeetu Patel describes rationing tokens like headcount: a fixed pool and an expected output. That matches measuring cost per completed task and watching context on every turn, themes that show up across [AI infrastructure](https://chainofthought.show/topics/ai-infrastructure/) conversations on the show.\n\nThe longer explainer, with the episode clips, is on [Chain of Thought](https://chainofthought.show/ai-decoded/what-is-a-token-in-ai/). It draws on [this episode](https://chainofthought.show/podcast/64-stop-token-maxxing-find-where-ai-actually-pays-off-jiaona-zhang/).\n\n[Subscribe to the Chain of Thought newsletter](https://newsletter.chainofthought.show/subscribe?utm_source=devto&utm_medium=referral&utm_campaign=newsletter) for new episodes and write-ups like this one.\n\n*Drafted with AI assistance from the episode transcripts.*", "url": "https://wpnews.pro/news/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you", "canonical_source": "https://dev.to/conorbronsdon/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you-2123", "published_at": "2026-10-27 15:30:14+00:00", "updated_at": "2026-10-27 15:42:17.896858+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "mlops"], "entities": ["Anthropic", "OpenAI", "Cisco", "Jeetu Patel", "Claude"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you", "markdown": "https://wpnews.pro/news/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you.md", "text": "https://wpnews.pro/news/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you.txt", "jsonld": "https://wpnews.pro/news/count-tokens-before-you-ship-so-agent-bills-do-not-surprise-you.jsonld"}}