Count tokens before you ship so agent bills do not surprise you

A developer published a practical guide arguing that teams should measure token consumption before shipping agent features, since context windows, per-token pricing and latency all hinge on the same unit. The writeup notes that chat history, tool definitions and tool results all consume the context budget and that agents multiply costs across planning, tool calls and retries, and it offers a sketch cost function that sums input and output tokens across every call. It cites Anthropic's price list, where output costs five times input, and Cisco president Jeetu Patel's argument that "price per token could turn into a vanity metric" while the price of the outcome is what matters.

Your API dashboard shows a spike, but the feature "felt small." Look at the tokens: what the model reads, writes, and re-sends on every step. Tokenization https://chainofthought.show/glossary/tokenization/ is not academic. Context limits, per-token pricing, and latency all use the same unit, and dense code or odd formatting can cost more than the word count suggests. Language models do not read whole words or raw characters. They read tokens, sub-word chunks from a fixed vocabulary. In English, a common word is often one token; rare words and code can split into several, and punctuation and spaces count too. OpenAI's rule of thumb is about four characters per token, or about three-quarters of a word, so 100 tokens is roughly 75 words. That ratio drifts. Non-English text, code, and weird formatting usually burn more tokens per visible line. Before you argue about model tier, paste representative prompts into a tokenizer and note input tokens per request. Guessing from word count is how teams under-budget chat history and tool output. The context window https://chainofthought.show/glossary/context-window/ is a token budget for one response, and the latest user message is only one part of it. System instructions, prior turns, tool definitions, and tool results all sit in that budget. Output tokens need room too. In chat, unless your app trims or summarizes, the full history goes back on the next turn, so input tokens climb every step. Exceeding the limit is not one universal behavior. Depending on the API, the request may be rejected or generation may stop, and many applications truncate or compact the input before sending it. Compaction changes what the model sees next; it does not make the window bigger. Anthropic's documentation also warns that accuracy and recall can degrade as context grows, what it calls context rot. For cost planning, whether a prompt fits in the window and whether it is cheap to keep sending are different questions. For most teams on hosted models, inference https://chainofthought.show/glossary/inference/ is the bill that repeats: every message and agent step is another run. Training is someone else's capex; your product pays per call. Providers charge input and output separately. On Anthropic's price list as of October 2026, output costs five times as much as input for each Claude model. Reasoning models can bill hidden reasoning as output, so a short visible answer can still be expensive. Repeated static instructions may qualify for prompt caching: Anthropic charges about a tenth of normal input to read a cached block, with a higher cost the first time you write it. One successful task is rarely one call. Agents plan, call tools, retry, and often carry a growing context each time. Cost per task is roughly the sum over every call of input tokens × input price plus output tokens × output price , including failures and retries. Jeetu Patel, Cisco's president and chief product officer, argued in episode 67 that price per token could turn into a vanity metric: the price per token falls while long-running agents use more tokens. What matters to him is the price of the outcome. Your estimator should still start with tokens, because that is what you control in design. Use your provider's current input and output rates. If you only remember the Anthropic ratio above, set OUTPUT PRICE = 5 INPUT PRICE as a placeholder until you paste real numbers from the price list. Label this a sketch, not production accounting: bash Sketch: monthly inference cost from volume. Prices are $ per 1M tokens; replace the placeholders with your provider's list. def monthly cost tasks per month, calls per task, avg input tokens, avg output tokens, input price, output price : calls = tasks per month calls per task per call = avg input tokens input price + avg output tokens output price / 1 000 000 return calls per call INPUT PRICE = 1.0 placeholder OUTPUT PRICE = 5 INPUT PRICE the ratio on Anthropic's list 50k tasks, 8 model calls each, 12k tokens in and 800 out per call bill = monthly cost 50 000, 8, 12 000, 800, INPUT PRICE, OUTPUT PRICE Run three scenarios on the same workflow: today's averages, a heavy month longer history, more retries , and a trimmed prompt fewer input tokens or fewer calls per task . The line that moves the total most is where you spend engineering time. Set calls per task realistically for agents. A single user action might be dozens of inferences. Divide total cost by completed tasks, not by API calls, when you compare features. I think that today a lot of organizations are not thinking about their token spend regularly and they're getting hit with their bills. Jiaona Zhang, Laurel Jiaona Zhang https://chainofthought.show/guests/jiaona-zhang/ ties that to "token maxxing": mandating AI use without asking whether the task earned the tokens, like using a model to redo slide fonts when one click would do. The estimator is how you catch that before finance does. If output tokens dominate long answers, reasoning traces, verbose tool JSON , tighten schemas, cap completion length, or route easy steps to a smaller model. If input tokens dominate system prompt, full chat log, fat tool results , trim history, summarize older turns, cache stable instructions, and stop pasting whole documents when a chunk will do. Jeetu Patel describes rationing tokens like headcount: a fixed pool and an expected output. That matches measuring cost per completed task and watching context on every turn, themes that show up across AI infrastructure https://chainofthought.show/topics/ai-infrastructure/ conversations on the show. The longer explainer, with the episode clips, is on Chain of Thought https://chainofthought.show/ai-decoded/what-is-a-token-in-ai/ . It draws on this episode https://chainofthought.show/podcast/64-stop-token-maxxing-find-where-ai-actually-pays-off-jiaona-zhang/ . Subscribe to the Chain of Thought newsletter https://newsletter.chainofthought.show/subscribe?utm source=devto&utm medium=referral&utm campaign=newsletter for new episodes and write-ups like this one. Drafted with AI assistance from the episode transcripts.