The idea of “getting value from every token” sounds like a sensible one, and I am starting to hear it more and more, both at work and in the world at large. On the surface it sounds very obviously right - don’t waste money, don’t burn through your tokens pointlessly. But underneath that “obvious” insight there is one big bad assumption - that token efficiency is a meaningful proxy for value. And tokens are particularly tempting because they are so easy to measure. Put them on a dashboard, track them over time, set a target, and suddenly lower token usage starts to look like success. But just because something is easy to count does not mean it is measuring the thing you actually care about.
The Software-Cost Paradox
AI makes producing software considerably cheaper, yet software products are not getting cheaper to buy. This seems like a contradiction, but software “cost” is made up of multiple parts - development cost, operating costs and customer pricing. AI is making the development cost cheaper, but that doesn’t mean a vendor is going to charge any less for it - if an app built by humans was priced at $10/month, why would the same app built by AI not also charge $10/month?
More importantly, when development gets cheap and easy, expectations rise. Imagine you had an app spec from 2024, in theory building that exact app now in 2026 with AI should be substantially cheaper because it requires less engineering effort. But realistically, this is never what happens - the spec is never frozen. Instead, the increased developer productivity from working with AI gets swallowed up by increased ambition, and the 2024 app spec is now 10x longer and the app gets custom reporting or third party integrations that were never in scope before.
The same thing is happening with AI tokens. Model inference is getting cheaper every day, but this does not mean that companies’ token spend will also be getting proportionally cheaper. Token cost reduction can encourage what was previously considered “uneconomic” uses of AI - they are more viable now because the overall cost has not increased. Alternatively, we may maintain a constant AI spend but actually produce more useful work because there are more available tokens in the budget. Falling token cost can actually increase overall consumption because it makes AI seem so cheap it’s almost negligible, so it gets used (for good and bad) in far more places than it was in the past.
Why “value in every token” is the Wrong Framing
It seems obvious to say, but a token has no intrinsic business value - it’s like any other input cost like compute, storage or network bandwidth. Token usage can tell you something useful about what a workflow costs, but it can’t tell you how well that workflow performed. Cost metrics have a nasty habit of quietly turning into performance metrics as soon as they are put on a dashboard, so the distinction is important. Trying to add a value to every token encourages questionable local workflow optimisation, for example enforcing smaller context windows, using fewer tool calls, and less exploration of either the existing codebase or alternative solutions. These all act to reduce an individual’s token usage, and while these strategies may be sensible under some circumstances, none of them necessarily produce a better overall result outside of the session appearing “cheaper”.
Many of the apparently “wasteful” tokens are doing valuable work. Reading what appears to be an irrelevant file looks like a waste, but exploration is a crucial part of finding out what is and is not relevant. A second pass at a solution costs tokens, as does a validation pass, or a self review, or reading some additional context, but anyone who actually uses AI on a daily basis knows that any one of those things could be the thing that catches a dodgy assumption or prevents a nasty bug. If you only consider the successful final output as valuable, you are hiding all the work that was required to get to that output reliably. Even worse, - it is often impossible to identify which individual tokens were even “useful”. The file that turned out to be irrelevant still told the agent what not to change, the test that passed produced no new feature, but increased confidence that nothing broke. Sometimes the value is in the tokens that produced nothing.
Have You Tried Working With Humans?
We already understand this from many years of working with actual human engineers - nobody sensible evaluates developers purely on their visible output per $ of salary. We ditched “lines of code” as a productivity metric forever ago, and with good reason. There is an entire industry built on quantifying developer productivity, trying to infer what productive work actually looks like from things we can measure like commits, pull requests, cycle time, deployments, review time, lead time, incidents and combinations of all them. We do this because the thing we really want to know - “how much useful engineering work did we do?” - is incredibly difficult to measure directly.
If this wasn’t true, no one would hire senior engineers, every company would be stuffed with cheap juniors producing line after line after line… But we do hire seniors, because we know what we are paying them the big bucks for - judgement, experience, knowing what not to change, spotting risky assumptions, anticipating failure modes… Most of this work is invisible because the only way it shows up is in the number of problems that never happened (try and make a metric for that!), but that’s where the cost is going: assurance. This is part of what makes token usage so seductive by comparison. With AI tokens, all work comes with a beautifully precise number attached to it - we know exactly how many tokens went in and out and exactly what they cost. After decades of attempting to quantify human developer productivity it’s very tempting to treat that number as something more meaningful than it is. Precision != relevance.
The same principle applies to AI agents - a short, cheap session might produce a reasonable looking solution whereas a more expensive one may burn tokens inspecting the existing codebase, running additional tests, reconsidering assumptions, or reviewing its own diff and retry when something looks wrong. If they both end up with the same feature, the second one looks less efficient if you measure only tokens. But as with the human engineers, what those extra tokens are buying you is assurance in the solution, lower risk, less manual validation and fewer failures down the line. We do not ask whether every minute of a human senior engineer’s time directly produced a visible output, we should be similarly cautious doing so with AI agents.
The analogy here is not expensive agents == senior devs. Any agent can waste huge amounts of tokens going round in circles, just like humans can waste huge amounts of time on dead-end solutions. The point is just that cheaper does not automatically mean better value. Sometimes spending more is what gets you to the reliable answer in the first place.
Efficiency Does Still Matter
But we should be clear about what efficiency means and we are talking about: token efficiency, cost efficiency or value creation. Token efficiency can be defined as “can we achieve the same outcome for fewer tokens”, cost efficiency as “is the outcome worth the cost of producing it” and value creation as “what outcomes can we achieve now that used to be too difficult or expensive”. All three matter, but they are not interchangeable. If you are in the business of creating commercial software products that is actually useful to people, token efficiency should not be the main goal - it is something to keep an eye on and optimise for sure, but only once you know you are comparing equivalent outcomes. “Can we get the same result for fewer tokens?” is a useful question. “Can we use fewer tokens?” on its own really isn’t.
The Risk of Using Tokens as a KPI
This is not an argument for ignoring token usage entirely, there is still plenty of opportunity to waste tokens if you are not careful - pointless repetition, large and unnecessary contexts, infinite looping, bad architecture…
However turning token usage into a metric risks it becoming a target, and we all know good old Goodheart’s law: "When a measure becomes a target, it ceases to be a good measure". It’s not a great leap to imagine measuring tokens quickly starting to reward workflows or systems that shift cost elsewhere - more reliance on human testing and validation, more errors, lower confidence in outputs, avoidance of large tasks because they take more work therefore appear less efficient. It’s the token-based version of being penny-wise and pound-foolish, yeh you may save some minimal $$ on inference but you will be making up for that elsewhere in your process, and it’s likely in an area that is not as well measured.
And The Solution Is…
Instead of trying to match token spend directly to outputs and asking if we got maximum value out of every token, we should change the question. We can ask if the final outcome was worth the resources we spent producing it? Did the additional context, exploration, validation or retries make the result more useful, more reliable, or require less human effort afterwards? And only then: could we achieve an equivalent outcome more efficiently?
The word “equivalent” is doing a lot of work there. Saving 30% of the tokens while producing something that needs more review, fails more often, or can only tackle simple problems is not an efficiency improvement, it’s just moving the cost somewhere else. But measuring output quality is a topic worthy of its own post some day…
As inference gets cheaper, we have a choice. We can use that reduction to make today’s workflows cost a few pennies less, or we can spend those same pennies buying more context, more validation, more assurance and tackling problems that previously weren’t economically viable at all. There’s nothing wrong with managing token costs, but turning token usage into something that must always be as minimal as possible is a remarkably unambitious way to manage AI.
Tokens are a cost to manage, not a productivity metric to optimise. Their great advantage is that they are easy to measure, their great danger is that this makes them easy to mistake for value. The goal isn’t to get value from every token, it’s to get the most valuable outcome from the system as a whole.