{"slug": "why-is-your-ai-using-so-many-tokens", "title": "Why is your AI using so many tokens?", "summary": "A single \"hi\" message to an AI assistant with Drive and inbox connectors can produce an 18,000-token request, because the app resends a system prompt of a few thousand tokens, JSON-schema tool descriptions and the full conversation history on every turn, according to an analysis of Claude Sonnet 5.5 list prices from November 2026. The piece reports that after 6 turns a user has typed about 2,600 tokens while the model has received about 120,000, and that without caching the total grows with the square of the conversation's length. It notes that Anthropic's prompt cache lasts 5 minutes, with cache reads costing 0.05x normal input price and cache writes 1.25x, while output tokens — including hidden reasoning tokens billed at the output price — cost 4x-8x the input price on most models.", "body_md": "You type “hi” into an AI assistant, and without warning the usage meter shows 18,000 tokens. One of them is yours. Let’s try to see where the rest went.\n\n⚠️ Best viewed on desktop\n\n### What you see\n\nNew chat\n\n### What gets sent\n\n**0** tokens in context\n\nRequestnot sent yet\n\nBilled so far$0\n\nClaude Sonnet 5.5 list prices, Nov 2026\n\nThe AI chat app shows you a conversation. The model has no idea what that is.\n\nThe model sees a single block of text, rebuilt from scratch and sent in full every time you submit a message, and it forgets all of it the second the answer is sent.\n\nThe panel shows both sides: the chat interface you see, and what actually gets sent to the LLM.\n\nYou open a new chat with an assistant that can read your Drive and your inbox, and type `“hi”`.\n\nBefore your message, the app puts a **system prompt** in front of it. That is where the instructions live, like who the assistant is and what it must never do.\n\nA few thousand tokens usually, before your first message.\n\nThen come the **tools**. Every action the assistant can take (search your Drive, edit a document or draft an email) is described in the request with a name, a description and a JSON schema telling the agent how to call it.\n\nThe model can only call what it has been told exists, so all of that ships with every request.\n\nYour **“hi”** is now the smallest thing in an 18,000-token prompt. Every connector you add makes it bigger whether it’s used or not.\n\nIf you use Claude Code, `/context` shows you this breakdown for your own session.\n\nThe model replies with 11 tokens.\n\nOutput is the expensive part, **4x-8x** the input token price on most models.\n\nYou send a second message.\n\nThe model has no memory of your first request.\n\nEvery turn, the app sends the entire conversation again: tools, system prompt, your “hi”, its reply, and then your new message at the bottom.\n\nEvery chat API works this way.\n\nThe conversation lives in the client, and each request carries the whole history. The interface just pretends to blur the line between what the model does and what the application around it handles for you.\n\nKeep talking and each request is a little bigger than the one before.\n\nYou paste last quarter’s report, and it rides along on every request after that.\n\nAfter 6 turns you have typed ~2,600 tokens, and the model has received ~120,000 tokens.\n\nEvery request pays again for everything before it. Without caching, that total grows with the square of the conversation’s length.\n\nYou ask the model a complex question, which triggers some **reasoning**.\n\nThe chat shows a collapsed *“Thought for 21s”*. That was 2,100 tokens of text the model wrote *(to itself)* before answering.\n\nThinking tokens are ordinary output tokens. They are [billed](https://platform.claude.com/docs/en/build-with-claude/thinking-steering-and-cost) at the output price, even when the app shows you only a summary or nothing at all.\n\nMost modern models will [keep](https://platform.claude.com/docs/en/build-with-claude/thinking#thinking-block-preservation-by-model) that thinking in every turn of the conversation.\n\nThat is an additional 2,100 tokens you carry around, for output you don’t even see.\n\nAll of this would be painful (and prohibitively expensive) without caching.\n\nIn practice, the provider keeps your conversation in its memory for a little bit of time.\n\nWhen the next request starts with exactly the same tokens, that part is a cache hit.\n\nThe striped blocks are hits. On Claude Sonnet 5.5, a [cache read](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) costs 0.05x the normal input price.\n\nOnly what is new since the last request gets written to cache, at 1.25x.\n\nThe catch: cache has a short lifetime. For Anthropic, it is **5 minutes**, and every hit resets the clock, so it stays warm while you are replying.\n\nThen you go get a coffee…\n\nYou come back and ask where you were.\n\nThe cache is gone, so the whole thing is processed and written to cache again at 1.25x. That request costs about **15 times** the one before the coffee break.\n\nThe cache covers everything from the top of the prompt down to the first token that differs.\n\nSince tools are defined at the very top, changing them [invalidates the entire cache](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).\n\nYou connect Slack in the middle of the session. Its tool definitions land at the top of the prompt, and your cache is reset.\n\nYou want your agent to be capable and smart.\n\nYou ask it for a complex report. The assistant calls a tool, gets the result, and calls another one.\n\nEach step is a separate request, and each one re-sends everything before it plus the latest tool result.\n\nYou sent one message and the model received 4 requests.\n\nThe weekly updates it read are now the biggest block in the prompt.\n\nWhen an assistant works through your files, tool results are where the context goes.\n\nSo far the panel has been zoomed in. This is the same conversation drawn against a 200,000-token context size. This is the hard limit on how much the model can read in one request.\n\nNewer models go up to a million tokens.\n\nAn afternoon of report work later, reading spreadsheets, email threads and PDFs, the prompt has crept up to the dashed line, the compaction threshold.\n\nEvery document it read is still in there, sent again on every request.\n\nWhen the context crosses that threshold, the assistant **compacts** it.\n\nThe model writes a summary of the conversation so far, and the app replaces the older turns with that summary.\n\nTools and the system prompt stay where they are, and only the most recent messages survive next to the summary.\n\nMost agentic harnesses will do this automatically before the window fills.\n\nIn Claude Code, the **`/compact`** command triggers it manually.\n\nThe chat history isn’t impacted, so the UI lies to you. The model no longer has any of it. All it has is a 3,400-token summary of what happened.\n\nYou ask what the Q2 report said about the vendor contract.\n\nThe summary kept the conclusion and dropped the text, so the model can tell you what it decided back then but not what the report said.\n\nThe context window is the only memory the model has. Everything in it is paid for again on every request, and whatever gets compacted out is gone for the model while it stays on your screen.\n\nI hope that helped demystify how agents work and why you’re using so many tokens so fast.\n\nIf you have any questions, feel free to leave a comment below.", "url": "https://wpnews.pro/news/why-is-your-ai-using-so-many-tokens", "canonical_source": "https://blog.raed.dev/posts/context-window/", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-04 20:51:30.119030+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools", "ai-infrastructure"], "entities": ["Anthropic", "Claude Sonnet 5.5", "Claude Code"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/why-is-your-ai-using-so-many-tokens", "markdown": "https://wpnews.pro/news/why-is-your-ai-using-so-many-tokens.md", "text": "https://wpnews.pro/news/why-is-your-ai-using-so-many-tokens.txt", "jsonld": "https://wpnews.pro/news/why-is-your-ai-using-so-many-tokens.jsonld"}}