{"slug": "shorter-prompts-are-making-your-ai-agents-more-expensive", "title": "Shorter Prompts Are Making Your AI Agents More Expensive", "summary": "GitHub's test of shortening the shell output its coding agent sees made every response smaller but caused the agent to run extra recovery commands that dragged the full history along, so tasks took longer and cost more, according to The AI Corner. The newsletter reports agentic workloads consume roughly 1,000 times the tokens of ordinary prompting, and that OpenClaw creator Peter Steinberger's team spent $1.3 million on OpenAI API tokens in a single month. The piece argues less context is not the fix and points to Upwork's new MCP connector for Claude and ChatGPT, which returns a ranked shortlist of identity-verified freelancers across 180+ countries with escrow on every payment.", "body_md": "# Is There a Right Way to Use AI?\n\nAlmost everyone I know is using it wrong.\n\nThey treat it like Google. They ask it to change one sentence in a 2,000-word article. A friend of mine picks actual fights with his Claude Cowork. Then they wonder why they’re spending hundreds of dollars a month on tools that were supposed to make life cheaper.\n\nEnterprises do the same thing at scale. When the API bill gets scary, they shorten prompts, cap output, and switch to cheaper models.\n\nGitHub tested exactly that. They shortened the shell output their coding agent sees, and every response got smaller. But whenever the agent missed something, it ran another command to recover it, and each extra turn dragged the whole history along. **Tasks took longer and cost more.**\n\nLess context was never the fix. Here’s what is.\n\n*together with [Upwork](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3):*\n\nHere’s the part of every build I still can’t automate: the moment I need someone who has actually done this before.\n\nUntil last week, that meant 2 screens, Claude on one and a freelancer marketplace on the other, copy-pasting between them for an afternoon.\n\n#### Upwork’s new **[MCP connector for Claude and ChatGPT](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)** collapses that into one conversation. \n\n[MCP connector for Claude and ChatGPT](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)\n\nYou describe the outcome, and it comes back with a ranked shortlist of freelance experts and the reasoning behind each one.\n\nOne client had Claude read 50 profiles against his [job post](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3). It took minutes. He’d budgeted a week.\n\n▫️ [Identity-verified freelancers](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3) across 180+ countries\n\n▫️ Escrow on every payment, and you pay only when you hire\n\n## Table of Contents\n\n**1.**  The Bill Nobody Modelled\n\n**2.**  The Experiment That Breaks the Obvious Fix\n\n**3.**  The Four Places Context Goes to Rot\n\n**4.**  The Changes That Cost Nothing to Make\n\n**5.**  The Changes That Need a Test Behind Them\n\n**6.**  The Problem Underneath the Problem\n\n## **1. The Bill Nobody Modelled**\n\nAn agent does not cost more than a chatbot because it is cleverer but mostly because of everything that happens **behind the scenes**.\n\n### **A thousand times the tokens for work that looks the same**\n\nA **chatbot** sends a message and receives one back.\n\nAn **agent** plans, calls a tool, reads the result, decides the next move, calls another tool, drafts something, runs a test, revises.\n\nEach of those steps writes text into the context window, and all of it gets **resentful** on every turn that actually follows.\n\nThe attention cost scales with the **square** of sequence length, so a context that doubles does not cost twice as much.\n\nAgentic workloads land somewhere around a **thousand** times the token consumption of ordinary prompting.\n\nPeter Steinberger, who built OpenClaw, put a **number** on the top end. [His team spent](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) [$1.3 million](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) [on tokens in one month,](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) 603 billion of them, across 100 coding agents run by three people.\n\nThere is a recent position paper on token reduction from Harvard, MIT and Northeastern singles out OpenClaw and Codex as the systems worth studying, simply because of how many tokens they **burn** through.\n\n### **The arithmetic nobody runs before shipping**\n\nThis version actually applies to **everyone** else. At roughly $2.50 per million input tokens, a single run carrying 30,000 tokens of context costs seven and a half **cents**.\n\nTen thousand runs a day is **$750**. Call it **$22,500** a month, for input alone, before the model generates a single word of output.\n\nAnd big companies are already **feeling** it. **Uber** reportedly exhausted its entire 2026 AI budget in four months. **Microsoft** is reported to have ended its Claude Code licences after a pilot that began in December 2025.\n\nIn that survey of 2,500 decision-makers, the rollback rate reached **81%** at companies with mature governance frameworks, which is to say the organisations paying closest attention pulled back hardest.\n\nThe thing is, none of the decisions behind those bills was crazy on its own, which is why **nobody** saw it coming.\n\n## **2. The Experiment That Breaks the Obvious Fix**\n\nGitHub publishes its agent cost engineering, which makes it one of the few places where any of this gets **tested** rather than asserted.\n\n### **Shorter calls, longer tasks**\n\nThe team evaluated a utility called **Rust Token Killer**, which shortens shell output before a coding agent reads it, and ran it against their own agentic coding benchmarks.\n\nIt did its [job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3), the tool responses got **shorter**.\n\nThen, when the omitted text turned out to **matter**, the agent reopened the original output or reran the command to recover what was missing. Each recovery added a **turn**, and every added turn dragged the whole accumulated context along with it.\n\nOn average the task consumed more **tokens** and took longer. Completion rates held steady. So each call got cheaper, but the whole task got more **expensive**.\n\n### **The unit is the [finished job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)**\n\n[finished job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)\n\nBy counting tokens per tool call means measuring the wrong **thing**, once you see that, a lot of the usual advice stops making sense.\n\nA response 200 tokens shorter that triggers two recovery turns is a **loss**. A response that runs longer but lets the agent finish in one pass is a **win**.\n\nAnd it works the **other** way round, too. That token reduction paper argues attention is a finite **budget**. Bury a decisive fact in the middle of a 40,000 token window and the model pays less attention to it, a well known problem researchers call [“lost in the middle”](https://arxiv.org/abs/2307.03172)[.](https://arxiv.org/abs/2307.03172)\n\nReasoning models left **uncapped** have been measured spending over 15,000 tokens on maths problems a few hundred tokens would solve.\n\nOne method the paper cites, CoT-Influx, **pruned** the reasoning examples sitting inside a prompt, which freed room for more of them, and accuracy went up.\n\nSo cheaper and better aren’t really **opposites**. You only lose quality when you cut something the system actually needed, so the first step is figuring out where the **wasted** tokens are.\n\n## Table of Contents\n\n**1.**  The Bill Nobody Modelled\n\n**2.**  The Experiment That Breaks the Obvious Fix\n\n**3.**  The Four Places Context Goes to Rot\n\n**4.**  The Changes That Cost Nothing to Make\n\n**5.**  The Changes That Need a Test Behind Them\n\n**6.**  The Problem Underneath the Problem\n\n## **1. The Bill Nobody Modelled**\n\nAn agent does not cost more than a chatbot because it is cleverer but mostly because of everything that happens **behind the scenes**.\n\n### **A thousand times the tokens for work that looks the same**\n\nA **chatbot** sends a message and receives one back.\n\nAn **agent** plans, calls a tool, reads the result, decides the next move, calls another tool, drafts something, runs a test, revises.\n\nEach of those steps writes text into the context window, and all of it gets **resentful** on every turn that actually follows.\n\nThe attention cost scales with the **square** of sequence length, so a context that doubles does not cost twice as much.\n\nAgentic workloads land somewhere around a **thousand** times the token consumption of ordinary prompting.\n\nPeter Steinberger, who built OpenClaw, put a **number** on the top end. [His team spent](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) [$1.3 million](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) [on tokens in one month,](https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-creator-burns-through-1-3-million-in-openai-api-tokens-in-a-single-month) 603 billion of them, across 100 coding agents run by three people.\n\nThere is a recent position paper on token reduction from Harvard, MIT and Northeastern singles out OpenClaw and Codex as the systems worth studying, simply because of how many tokens they **burn** through.\n\n### **The arithmetic nobody runs before shipping**\n\nThis version actually applies to **everyone** else. At roughly $2.50 per million input tokens, a single run carrying 30,000 tokens of context costs seven and a half **cents**.\n\nTen thousand runs a day is **$750**. Call it **$22,500** a month, for input alone, before the model generates a single word of output.\n\nAnd big companies are already **feeling** it. **Uber** reportedly exhausted its entire 2026 AI budget in four months. **Microsoft** is reported to have ended its Claude Code licences after a pilot that began in December 2025.\n\nIn that survey of 2,500 decision-makers, the rollback rate reached **81%** at companies with mature governance frameworks, which is to say the organisations paying closest attention pulled back hardest.\n\nThe thing is, none of the decisions behind those bills was crazy on its own, which is why **nobody** saw it coming.\n\n## **2. The Experiment That Breaks the Obvious Fix**\n\nGitHub publishes its agent cost engineering, which makes it one of the few places where any of this gets **tested** rather than asserted.\n\n### **Shorter calls, longer tasks**\n\nThe team evaluated a utility called **Rust Token Killer**, which shortens shell output before a coding agent reads it, and ran it against their own agentic coding benchmarks.\n\nIt did [its job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3), the tool responses got **shorter**.\n\nThen, when the omitted text turned out to **matter**, the agent reopened the original output or reran the command to recover what was missing. Each recovery added a **turn**, and every added turn dragged the whole accumulated context along with it.\n\nOn average the task consumed more **tokens** and took longer. Completion rates held steady. So each call got cheaper, but the whole task got more **expensive**.\n\n### **The unit is the [finished job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)**\n\n[finished job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3)\n\nBy counting tokens per tool call means measuring the wrong **thing**, once you see that, a lot of the usual advice stops making sense.\n\nA response 200 tokens shorter that triggers two recovery turns is a **loss**. A response that runs longer but lets the agent finish in one pass is a **win**.\n\nAnd it works the **other** way round, too. That token reduction paper argues attention is a finite **budget**. Bury a decisive fact in the middle of a 40,000 token window and the model pays less attention to it, a well known problem researchers call [“lost in the middle”](https://arxiv.org/abs/2307.03172)[.](https://arxiv.org/abs/2307.03172)\n\nReasoning models left **uncapped** have been measured spending over 15,000 tokens on maths problems a few hundred tokens would solve.\n\nOne method the paper cites, CoT-Influx, **pruned** the reasoning examples sitting inside a prompt, which freed room for more of them, and accuracy went up.\n\nSo cheaper and better aren’t really **opposites**. You only lose quality when you cut something the system actually needed, so the first step is figuring out where the **wasted** tokens are.\n\n## **3. The Four Places Context Goes to Rot**\n\nWaste concentrates in four reservoirs, and each one wants a different fix.\n\n### **Prompts and tool output**\n\nThe system prompts tend to **grow** over time, because people keep adding instructions and almost never take any out. A prompt that started at 500 tokens arrives at **5,000**, and most sit at roughly three times the length they need.\n\nAgent harnesses make it even **messier**. GitHub found the guidance for a single task tool scattered across tool descriptions, schemas, agent definitions, system instructions and companion tools, and no one was in **charge** of the full picture.\n\nTool output is worse, because it arrives **raw**.\n\nOne web search result runs 2,000 to 8,000 tokens of stripped text. A 50 page policy document loaded in as reference material is 25,000 to 40,000 tokens, and any given query needs about three **paragraphs** of it.\n\n### **History and thinking**\n\n[Ten turns into a conversation, history alone is](https://www.trychroma.com/research/context-rot) [15,000](https://www.trychroma.com/research/context-rot) [tokens and climbing.](https://www.trychroma.com/research/context-rot) By turn twenty an agent can be sending 30,000 tokens of past exchanges just to ask what happened in the second one.\n\nDevelopers often make this **worse** on purpose, writing recent actions, errors and state into the prompt so the agent remembers what it has done.\n\nAnd then there’s the **thinking** part. Extended reasoning modes add 5,000 to 20,000 tokens to calls that often need none of it, and if you don’t put a **cap** on it, the model will happily use all of it.\n\nThree of those four drain without any risk to output quality at all.\n\n## **4. The Changes That Cost Nothing to Make**\n\nSome changes leave the model receiving **precisely** what it received before. Those go first, because there is nothing to weigh up.\n\n### **Caching, and the ordering that makes it work**\n\nMarking the stable part of a prompt as cacheable drops its price to around a **tenth** of the normal input rate. Writing to the cache costs roughly 25% extra, so you break even after about two calls.\n\nBasically, anything you send more than once is worth **caching**.\n\nThe **order** of your prompt is what makes or breaks this. Static content goes first, meaning the system prompt, examples, reference documents and **schemas**.\n\nDynamic content goes after, meaning the query, the tool results, anything that **varies**. OpenAI caches prefixes automatically above 1,024 tokens,\n\nAnthropic uses explicit **breakpoints**, and the cache lasts about five minutes by default.\n\nOne research agent sending a 6,000 token system prompt on every call saw input costs fall **87%** on this change alone.\n\nAgents sharing a large system prompt across many users typically save between 60% and 80% on the cached **portion**.\n\nThe **catch** is that a single dynamic detail, like a timestamp, a session ID or a user name, can slip into the static block without anyone noticing.\n\nWhen it does, everything after it gets **reprocessed** at full price.\n\n### **Formatting nobody uses, files nobody needs**\n\nGitHub’s file reading tool had been adding a **number** to the start of every line of output. Older editing tools needed those numbers to target changes.\n\nCurrent ones match the surrounding **code** instead, so the numbers had been dead weight for a while.\n\nRemoving them cut inference cost around **5%** in offline benchmarks and about 3% per user per day in production, with success rates inside normal variance and no increase in edit failures.\n\nPDFs have a similar hidden **cost**. Every page gets rendered as an image and processed as visual plus text tokens, costing 1,500 to 3,000 tokens even when it holds 200 words.\n\nThose same words pasted as plain text run about **250**. One 42 page report cost **84,000** tokens per call as a PDF and 9,500 after conversion.\n\nAnd there’s a **fourth** one, hidden in how the agent juggles its tasks. When a background task finishes and the completion notice doesn’t include the result, the agent wastes a whole turn asking for something the harness already has.\n\nTwo tasks running at the same time meant **four** model calls before work could resume. Batching those completions cut it down to **one**, which saved about 2.3% without compressing, summarising or leaving out anything.\n\nFrom here on, the changes need some **judgment**, plus a way to test them.\n\n## **5. The Changes That Need a Test Behind Them**\n\nThe remaining levers are **bigger**, and all of them change what the model sees.\n\n### **Difficulty is the cheapest thing to measure**\n\nSending every request to the strongest available model is the equivalent of paying a surgeon’s hourly rate to apply a plaster.\n\nMost tasks can be sorted with **5 simple questions**.\n\n- Is the work repetitive, or does it need real **judgment** ?\n- If a mistake slips through, will someone catch it later, or will it spread?\n- Does the task need actual **reasoning** , or just reshaping what’s already there?\n- Will a customer see the output?\n- Does [the job](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3) need the best possible answer, or just one that works?\n\nMost of the time, a simple **rule** is enough to decide which model gets the task\n\nWhen it isn’t, a call to the cheapest model costs almost nothing next to what it **saves**. A simple classifier pushes 60% to 70% of production traffic into the bottom **tier**, where tokens run 10 to 50 times cheaper.\n\nOne team shipped a router and watched the monthly bill fall **58%** that same week.\n\nThinking budgets work the same way. Lookups run fine at zero thinking tokens, moderate reasoning at 1,000 to 2,000, and really hard analysis at **10,000 to 16,000**.\n\nTeams that adjust the budget to each query report 50% to 75% off their thinking token **cost**, because most production queries are routine.\n\n### **Compression that survives contact with an agent**\n\nGitHub’s shipped compressor ended up **conservative**, and its shape is worth copying.\n\nAnything that looks like source code or unpredictable output is left **untouched**, meaning file contents, diffs and script results.\n\nThose are exactly the places where one missing line sends the agent straight back. Search results get reorganised without losing anything, and every **match** is kept.\n\nOnly predictable noise gets compressed, meaning install logs, build output, test runners and progress bars, and only when it saves a meaningful **amount**.\n\n Git diff was in the compression set at first. It was taken out after benchmark tasks showed agents reopening the **originals**.\n\n The agent can always go back to the full original, and how often it does is a better sign of whether compression works than any offline **metric**.\n\nWhen compression is safe, the savings are **big**. Semantic compression cuts oversized tool output by 70% to 90%.\n\nRolling summarisation keeps the last four or five turns word for word and folds the rest into a short **memory** block, which takes a 20 turn history from 8,000 to 15,000 tokens down to under 2,000.\n\nMoving logs and state into **SQLite** and pulling them only when needed swaps 6,000 tokens of carried history for 300.\n\nThere’s one more story from the same GitHub work that’s worth **telling**. They used an automated loop to rewrite an agent prompt, which cut it to half its **size**, passed offline evaluation, and shipped.\n\n[But in production, subagents that used to run at the same time started running one after another.](https://www.the-ai-corner.com/p/claude-code-dynamic-workflows-6-patterns-14-steps-anthropic-engineers-2026) The rewrite had taken a gentle suggestion about running tasks in parallel and turned it into a strict **rule**.\n\nThey stopped the experiment and wrote a test to catch the behaviour users had spotted. The final fix was a single sentence that was both shorter and less **restrictive** than the original.\n\n**An untested behaviour can be deleted by a compression pass without anyone noticing.**\n\n## **6. The Problem Underneath the Problem**\n\nEverything above is about making an existing system cheaper. But a lot of agent costs come from systems that were **built** the wrong way in the first place.\n\nBefore you hand a workflow to an agent, you should be able to answer five **questions**. What does it receive? What exactly does it do? What does it produce? Who checks the result? And how does it know when it’s finished?\n\nIf you leave any of them blank, the agent will just **guess**. [Without a stop condition](https://www.the-ai-corner.com/p/loop-engineering-coding-agents-2026), it keeps rewriting work that was already fine.\n\nWithout a review point, it runs steps that change nothing. Without a clear output, agents keep passing work back and forth forever. From the outside it all looks **productive**, and you pay for every token of it.\n\nContext works the same way. An agent writing headlines doesn’t need the brand wiki. An agent fixing formatting doesn’t need the company **strategy**.\n\n[Most of the work is giving each agent a small pack of context](https://www.the-ai-corner.com/p/ai-agent-memory-context-as-topology-playbook-2026) for its own **role**, instead of handing everyone the same knowledge base.\n\nAnd that’s the real **problem**. Most runaway token bills aren’t an efficiency issue at all, they’re a design issue. Teams resend huge histories, pull in too much data, default to the biggest model and ask for long answers, all because nobody ever decided what mattered.\n\nThe system doesn’t know what’s important, so it sends **everything**.\n\nIt’s also why measuring AI adoption by how much people use it tends to **backfire**.\n\n[Leaderboards that celebrate whoever burned the most tokens](https://www.hcamag.com/us/specialization/hr-technology/amazon-shuts-down-ai-leaderboard-after-tokenmaxxing/577189) reward **activity** over good judgment, and you end up with exactly what you measured.\n\nThat means premium models doing trivial work, agents used where one prompt would have done, and ten step workflows built for two step **problems**.\n\nThe teams that got this right didn’t just send fewer tokens, they **chose** which ones to send.\n\n**An agent’s invoice was never a record of how hard it worked. It is a record of everything nobody bothered to decide.**\n\n*from our partners:* \n\nGitHub's lesson applies to hiring too. The expensive move is skimping on the scope and paying for it in rounds of interviews.\n\nDescribe the outcome once, and Upwork's [MCP connector](https://www.upwork.com/ai/plugins?utm_medium=PaidSocial&utm_source=substack&utm_campaign=PAIDSOCIAL_substack_INFL_US_MP_CONS_INTEREST_CL_MCPPlugin_theaicornerQ3&utm_keyword=theaicornerQ3) lets Claude rank the freelance experts who fit, with the reasoning attached. \n\n## **3. The Four Places Context Goes to Rot**\n\nWaste concentrates in four reservoirs, and each one wants a different fix.\n\n### **Prompts and tool output**\n\nThe system prompts tend to **grow** over time, because people keep adding instructions and almost never take any out. A prompt that started at 500 tokens arrives at **5,000**, and most sit at roughly three times the length they need.\n\nAgent harnesses make it even **messier**. GitHub found the guidance for a single task tool scattered across tool descriptions, schemas, agent definitions, system instructions and companion tools, and no one was in **charge** of the full picture.\n\nTool output is worse, because it arrives **raw**.\n\nOne web search result runs 2,000 to 8,000 tokens of stripped text. A 50 page policy document loaded in as reference material is 25,000 to 40,000 tokens, and any given query needs about three **paragraphs** of it.\n\n### **History and thinking**\n\n[Ten turns into a conversation, history alone is](https://www.trychroma.com/research/context-rot) [15,000](https://www.trychroma.com/research/context-rot) [tokens and climbing.](https://www.trychroma.com/research/context-rot) By turn twenty an agent can be sending 30,000 tokens of past exchanges just to ask what happened in the second one.\n\nDevelopers often make this **worse** on purpose, writing recent actions, errors and state into the prompt so the agent remembers what it has done.\n\nAnd then there’s the **thinking** part. Extended reasoning modes add 5,000 to 20,000 tokens to calls that often need none of it, and if you don’t put a **cap** on it, the model will happily use all of it.\n\nThree of those four drain without any risk to output quality at all.\n\n## **4. The Changes That Cost Nothing to Make**\n\nSome changes leave the model receiving **precisely** what it received before. Those go first, because there is nothing to weigh up.\n\n### **Caching, and the ordering that makes it work**\n\nMarking the stable part of a prompt as cacheable drops its price to around a **tenth** of the normal input rate. Writing to the cache costs roughly 25% extra, so you break even after about two calls.\n\nBasically, anything you send more than once is worth **caching**.\n\nThe **order** of your prompt is what makes or breaks this. Static content goes first, meaning the system prompt, examples, reference documents and **schemas**.\n\nDynamic content goes after, meaning the query, the tool results, anything that **varies**. OpenAI caches prefixes automatically above 1,024 tokens,\n\nAnthropic uses explicit **breakpoints**, and the cache lasts about five minutes by default.\n\nOne research agent sending a 6,000 token system prompt on every call saw input costs fall **87%** on this change alone.\n\nAgents sharing a large system prompt across many users typically save between 60% and 80% on the cached **portion**.\n\nThe **catch** is that a single dynamic detail, like a timestamp, a session ID or a user name, can slip into the static block without anyone noticing.\n\nWhen it does, everything after it gets **reprocessed** at full price.\n\n### **Formatting nobody uses, files nobody needs**\n\nGitHub’s file reading tool had been adding a **number** to the start of every line of output. Older editing tools needed those numbers to target changes.\n\nCurrent ones match the surrounding **code** instead, so the numbers had been dead weight for a while.\n\nRemoving them cut inference cost around **5%** in offline benchmarks and about 3% per user per day in production, with success rates inside normal variance and no increase in edit failures.\n\nPDFs have a similar hidden **cost**. Every page gets rendered as an image and processed as visual plus text tokens, costing 1,500 to 3,000 tokens even when it holds 200 words.\n\nThose same words pasted as plain text run about **250**. One 42 page report cost **84,000** tokens per call as a PDF and 9,500 after conversion.\n\nAnd there’s a **fourth** one, hidden in how the agent juggles its tasks. When a background task finishes and the completion notice doesn’t include the result, the agent wastes a whole turn asking for something the harness already has.\n\nTwo tasks running at the same time meant **four** model calls before work could resume. Batching those completions cut it down to **one**, which saved about 2.3% without compressing, summarising or leaving out anything.\n\nFrom here on, the changes need some **judgment**, plus a way to test them.\n\n## **5. The Changes That Need a Test Behind Them**\n\nThe remaining levers are **bigger**, and all of them change what the model sees.\n\n### **Difficulty is the cheapest thing to measure**\n\nSending every request to the strongest available model is the equivalent of paying a surgeon’s hourly rate to apply a plaster.\n\nMost tasks can be sorted with **5 simple questions**.\n\n- Is the work repetitive, or does it need real **judgment** ?\n- If a mistake slips through, will someone catch it later, or will it spread?\n- Does the task need actual **reasoning** , or just reshaping what’s already there?\n- Will a customer see the output?\n- Does the job need the best possible answer, or just one that works?\n\nMost of the time, a simple **rule** is enough to decide which model gets the task\n\nWhen it isn’t, a call to the cheapest model costs almost nothing next to what it **saves**. A simple classifier pushes 60% to 70% of production traffic into the bottom **tier**, where tokens run 10 to 50 times cheaper.\n\nOne team shipped a router and watched the monthly bill fall **58%** that same week.\n\nThinking budgets work the same way. Lookups run fine at zero thinking tokens, moderate reasoning at 1,000 to 2,000, and really hard analysis at **10,000 to 16,000**.\n\nTeams that adjust the budget to each query report 50% to 75% off their thinking token **cost**, because most production queries are routine.\n\n### **Compression that survives contact with an agent**\n\nGitHub’s shipped compressor ended up **conservative**, and its shape is worth copying.\n\nAnything that looks like source code or unpredictable output is left **untouched**, meaning file contents, diffs and script results.\n\nThose are exactly the places where one missing line sends the agent straight back. Search results get reorganised without losing anything, and every **match** is kept.\n\nOnly predictable noise gets compressed, meaning install logs, build output, test runners and progress bars, and only when it saves a meaningful **amount**.\n\n Git diff was in the compression set at first. It was taken out after benchmark tasks showed agents reopening the **originals**.\n\n The agent can always go back to the full original, and how often it does is a better sign of whether compression works than any offline **metric**.\n\nWhen compression is safe, the savings are **big**. Semantic compression cuts oversized tool output by 70% to 90%.\n\nRolling summarisation keeps the last four or five turns word for word and folds the rest into a short **memory** block, which takes a 20 turn history from 8,000 to 15,000 tokens down to under 2,000.\n\nMoving logs and state into **SQLite** and pulling them only when needed swaps 6,000 tokens of carried history for 300.\n\nThere’s one more story from the same GitHub work that’s worth **telling**. They used an automated loop to rewrite an agent prompt, which cut it to half its **size**, passed offline evaluation, and shipped.\n\n[But in production, subagents that used to run at the same time started running one after another.](https://www.the-ai-corner.com/p/claude-code-dynamic-workflows-6-patterns-14-steps-anthropic-engineers-2026) The rewrite had taken a gentle suggestion about running tasks in parallel and turned it into a strict **rule**.\n\nThey stopped the experiment and wrote a test to catch the behaviour users had spotted. The final fix was a single sentence that was both shorter and less **restrictive** than the original.\n\n**An untested behaviour can be deleted by a compression pass without anyone noticing.**\n\n## **6. The Problem Underneath the Problem**\n\nEverything above is about making an existing system cheaper. But a lot of agent costs come from systems that were **built** the wrong way in the first place.\n\nBefore you hand a workflow to an agent, you should be able to answer five **questions**. What does it receive? What exactly does it do? What does it produce? Who checks the result? And how does it know when it’s finished?\n\nIf you leave any of them blank, the agent will just **guess**. [Without a stop condition](https://www.the-ai-corner.com/p/loop-engineering-coding-agents-2026), it keeps rewriting work that was already fine.\n\nWithout a review point, it runs steps that change nothing. Without a clear output, agents keep passing work back and forth forever. From the outside it all looks **productive**, and you pay for every token of it.\n\nContext works the same way. An agent writing headlines doesn’t need the brand wiki. An agent fixing formatting doesn’t need the company **strategy**.\n\n[Most of the work is giving each agent a small pack of context](https://www.the-ai-corner.com/p/ai-agent-memory-context-as-topology-playbook-2026) for its own **role**, instead of handing everyone the same knowledge base.\n\nAnd that’s the real **problem**. Most runaway token bills aren’t an efficiency issue at all, they’re a design issue. Teams resend huge histories, pull in too much data, default to the biggest model and ask for long answers, all because nobody ever decided what mattered.\n\nThe system doesn’t know what’s important, so it sends **everything**.\n\nIt’s also why measuring AI adoption by how much people use it tends to **backfire**.\n\n[Leaderboards that celebrate whoever burned the most tokens](https://www.hcamag.com/us/specialization/hr-technology/amazon-shuts-down-ai-leaderboard-after-tokenmaxxing/577189) reward **activity** over good judgment, and you end up with exactly what you measured.\n\nThat means premium models doing trivial work, agents used where one prompt would have done, and ten step workflows built for two step **problems**.\n\nThe teams that got this right didn’t just send fewer tokens, they **chose** which ones to send.\n\n**An agent’s invoice was never a record of how hard it worked. It is a record of everything nobody bothered to decide.**", "url": "https://wpnews.pro/news/shorter-prompts-are-making-your-ai-agents-more-expensive", "canonical_source": "https://www.the-ai-corner.com/p/cut-ai-agent-costs-without-losing-performance", "published_at": "2026-09-26 15:48:54+00:00", "updated_at": "2026-09-26 16:01:13.752416+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "large-language-models", "agent-protocols", "ai-tools"], "entities": ["GitHub", "OpenClaw", "Peter Steinberger", "OpenAI", "Upwork", "Claude", "ChatGPT", "The AI Corner"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/shorter-prompts-are-making-your-ai-agents-more-expensive", "markdown": "https://wpnews.pro/news/shorter-prompts-are-making-your-ai-agents-more-expensive.md", "text": "https://wpnews.pro/news/shorter-prompts-are-making-your-ai-agents-more-expensive.txt", "jsonld": "https://wpnews.pro/news/shorter-prompts-are-making-your-ai-agents-more-expensive.jsonld"}}