{"slug": "ai-tokenomics-how-to-tokenmin-while-roimaxxing", "title": "AI Tokenomics: How to tokenmin while ROImaxxing", "summary": "A company accidentally spent $500 million on AI tokens in a single month due to a lack of employee usage limits, while Uber and ServiceNow burned through their annual AI budgets in months. Major providers like Anthropic and GitHub Copilot are shifting to usage-based billing, increasing costs up to 18x, and enterprises deploying AI agents face 15x to 1000x higher token usage. Experts say 50-80% of token spend is unnecessary and recommend context management, multi-model systems, and workflow optimization to reduce costs.", "body_md": "A company [accidentally](https://www.axios.com/2026/05/28/ai-spending-roi-enterprise-costs) spent $500 million on tokens. We’re not making this up; fact is stranger than your AI agent’s wildest hallucination. And they managed to spend it all in *a single month* because they hadn’t set employee limits on Claude usage. In a similar vein, Uber and Service Now burned through their [entire annual AI budgets](https://www.theinformation.com/newsletters/applied-ai/uber-cto-shows-claude-code-can-blow-ai-budgets) in the first few months of this year.\n\nThese extraordinary levels of expenditure are driven by dramatic developments along two axes: price and quantity. Major AI providers like Anthropic and GitHub Copilot are moving from generous flat-rate subscriptions to usage-based billing. For example, GitHub Copilot’s June 2026 pricing changes [increased model costs](https://docs.github.com/en/copilot/reference/copilot-billing/request-based-billing-legacy/model-multipliers-for-annual-plans) for legacy Pro and Pro+ users by as much as 9x to 18x. Also, enterprises are increasingly deploying AI agents at scale, and multi-agent systems [typically use](https://www.anthropic.com/engineering/multi-agent-research-system) about 15x more tokens than chat interactions (meanwhile, agentic coding systems [can use](https://arxiv.org/pdf/2604.22750) >1000x more tokens than chat). And let’s not forget the less obvious pricing/quantity changes, as seen when [Anthropic’s Opus 4.7 shipped with a tokenizer](https://www.finout.io/blog/anthropic-api-pricing) which kept per-token rates identical, but generates up to 35% more tokens from the same input.\n\nThe good news is that 50-80% of your token spend is unnecessary. There is a multitude of (hidden) ways in which you’re bleeding tokens: everything from using frontier models for trivial tasks, to re-processing the same context repeatedly, to having “gossipy” AI agents who send vast quantities of superfluous information back-and-forth. Reassuringly, all of these problems have fixes, which we discuss in this report.\n\nHere’s our blueprint for token-efficient AI:\n\n**Invest in context and memory management solutions**. Input tokens, rather than output tokens, account for the larger share of token spend. The best context and memory management solutions feed AI agents the precise information they need (not too little, not too much) in an efficient manner whilst improving accuracy. Also, because these solutions exist independently of model providers, it’s easier to switch models.**Build multi-model systems**. Model capabilities are evolving too quickly to bet on a single provider, and Anthropic’s “[Fable Fracas](https://www.anthropic.com/news/fable-mythos-access)” showed that access to frontier intelligence can vanish overnight for geopolitical reasons. In this report, we’ve focused on 3 key components of multi-model systems:- Open source models. As open and closed model performance increasingly converge, ensembles of budget models can rival or exceed frontier intelligence at around half the cost.\n- AI routers and gateways. For most of the use cases you don’t need frontier intelligence, so routers and gateways direct your traffic to the lowest cost model that can meet your performance requirements.\n- Inference providers. Agentic AI demands a new inference stack because unlike chat, agentic workloads are long-running, asynchronous, context-heavy, and dominated by tool use. That’s where async and batch inference can tremendously lower costs.\n\n**Make verification fast and cheap.** About 60% of the cost of agentic software engineering isn’t in initial code generation, but in automated refinement and verification. The “fully loaded” cost of tokens should also include the cost of human verification and re-work.**Focus on workflow design**. Even if you run the same agent on the same task, the cost can vary by 30x, and this unpredictability necessitates two actions:- Building real-time visibility into costs and ways to control that expenditure; and\n- Reserving LLMs for reasoning, not tasks that SQL, rules, or templates can handle. If something can be handled accurately and cheaply using deterministic methods, it’s best to avoid using agents.\n\n**Introduce spend controls without stifling adoption, by factoring in actual adoption patterns.** Use pooled budgets rather than fixed per-user limits to balance experimentation with cost discipline – e.g. 5% of users may account for 40% of token consumption, and this shifts around over time depending on the pace and depth of adoption.**Don’t optimise for token cost alone.** Security, governance and sovereignty are equally important considerations when designing AI systems.\n\nWe’ve also featured various startups and scaleups providing solutions (for everything from context and memory management to inference provision) in a market map. If you’re building in this space – or you’re an AI-native company cleverly ‘tokenminning’ while ‘ROImaxxing’ – please reach out to [Advika](https://mmc.vc/team/advika-jalan/), [Simon](https://mmc.vc/team/simon-menashy/) or [Prakriti](https://mmc.vc/team/prakriti-roy/) – we’d love to chat. We’ve been researching on and investing in AI companies for over a decade, and today we have one of the largest portfolios of AI companies in Europe – so we’re clearly very keen on the space.\n\n*Note: Certain startups and scaleups may fall into multiple categories and sub-categories; for the sake of simplicity we have assigned each company a single sub-category*.\n\n## Tech overview\n\nIn our previous work * Agentic Enablers: Treating AI’s amnesia and other disorders *we discussed context and memory management solutions at length.\n\n- Context refers to short-term memory that an AI agent has: the prompts you send, the files you attach, the tools you call.\n- Memory refers to long-term persistence – so you don’t have to constantly remind your AI agent key details about your business or how you want certain tasks to be performed.\n\nContext and memory management improve agent accuracy, which also means you’ll waste fewer tokens and less time fixing any incorrect outputs. Also, an independent context and memory management layer gives you much more flexibility in choice of model provider (this will be particularly important as you leverage cheaper, open source or smaller models – but we’re getting ahead of ourselves).\n\n“Companies want the data mediation layer to be independent from the model provider or agent harness – not only so they can switch between models, but also to enable all their agents to benefit from shared optimisations. It’s inherently better for them to have complete control over their data.”– Matt Henderson, CEO & Co-founder, Coral\n\n[Bai et al. (2026)](https://arxiv.org/pdf/2604.22750) found that input tokens rather than output tokens drive the overall cost of agentic coding tasks, and this is driven by the sheer quantity of tokens consumed. We note that once GPT-5.4 crosses 272K tokens, input prices double and output prices go up for the whole session, so a text-heavy agent can easily drive up the bills. That’s why it’s important to optimise context everywhere – from system prompt to tool calling to file formats to knowledge ingestion, as we detail below:\n\n**System prompt**: Before the agents even look at the user prompt, the system injects instructions defining their roles, constraints, and output formats. In many frameworks, these system prompts are sent *every single time* an agent is invoked. Tightening system prompts and moving rarely used guidance into RAG is helpful in [reducing input costs](https://www.requesty.ai/blog/maximize-ai-efficiency-how-prompt-caching-cuts-costs-by-up-to-a-staggering-90) by as much as 30%.\n\n**External knowledge or web search execution**: Poorly implemented web search can flood an agent’s context with full pages, repeated snippets, irrelevant results, ads, navigation text, duplicate sources and bloated citation metadata. The model then wastes tokens reading and reasoning over noise, increasing cost and latency while making answers less reliable. Search APIs solve this by returning cleaner, more structured and more relevant context from fresh, trusted sources. Instead of forcing agents to scrape and ingest messy webpages, they help surface the right information in a compact, deduplicated format. That improves accuracy while keeping token usage closer to the “Goldilocks” zone: enough context to avoid hallucination, not so much that performance degrades through context rot or inflated spend. Startups and scaleups building Search APIs include [Exa](https://exa.ai/), [Tavily](https://www.tavily.com/), [Parallel](https://parallel.ai/), [Cala](https://www.cala.ai/), [Linkup](https://www.linkup.so/), [Valyu](https://www.valyu.ai/), [Firecrawl](https://www.firecrawl.dev/), [Olostep](https://www.olostep.com/) and [Seltz](https://seltz.ai/).\n\n**Internal knowledge:** It’s equally important that internal knowledge is retrieved, curated and fed to the AI models to keep it in the Goldilocks zone we talked about, and here we’re seeing a fascinating variety of approaches emerging. We break these down into different sub-categories:\n\n**Retrieval infrastructure**: These platforms optimise the retrieval layer itself – how context is fetched and ranked before it reaches the model. Startups in this space include[PageIndex](https://pageindex.ai/)(vectorless RAG retrieval) and[Ragie](https://www.ragie.ai/)(a fully managed RAG-as-a-Service platform for developers).**Context and memory infrastructure**: The common thread across these companies is the shift from retrieval-heavy AI to structured, reusable context infrastructure. Rather than asking agents to repeatedly parse raw documents, SaaS data, tickets, transcripts, logs, execution traces, or employee know-how, startups like[Hyperspell](https://www.hyperspell.com/),[Modern Relay](https://modernrelay.com/),[Along AI](https://www.alongagents.ai/),[Clarifeye](https://www.clarifeye.ai/),[Saphenia](https://saphenia.com/),[Sentra](https://sentra.app/),[Cognee](https://www.cognee.ai/),[Mem0,](https://mem0.ai/)[Zep](https://www.getzep.com/),[Xmemory](https://xmemory.ai/),[Prometheux](https://prometheux.ai/),[Stardog](https://www.stardog.com/),[Datalinks](https://datalinks.com/)turn enterprise knowledge into persistent “company brains” through memory stores, context graphs, ontologies, typed facts, and secure knowledge spaces.[Edra](https://edra.ai/)turns real-world workflows into reusable playbooks for agents,[Signet AI](https://signetai.sh/)preserves an agent’s identity, memory, and secrets across models, and[Mubit](https://mubit.ai/)learns from past runs so agents can reuse successful approaches instead of reprocessing the same context.\n\n**File format:** Token costs quickly add up when agents pass around bulky formats like screenshots, verbose JSON, or entire email threads instead of simple text. Screenshots are especially expensive because models have to process every visual detail, while formats like JSON and HTML waste tokens on brackets, tags, and other formatting. The easiest fix is to pass the simplest format that still does the job – plain text, compact tables, or pre-parsed context wherever possible. Tools like [iGPT AI](https://github.com/igptai/) help by stripping out duplicate email chains, boilerplate, and other clutter before the model sees it, reducing token costs without losing the information that matters.\n\n**Tools gateway:** Poorly designed tools waste tokens because agents fail, retry, and carry around far more context than they need. Every extra tool and oversized API response adds to the prompt, increasing both cost and the chance of worse reasoning. The fix is to expose only the tools an agent needs, return concise summaries instead of raw payloads, and cache repetitive data. Over and above providing a tools connectivity and security/governance layer, [StackOne’s Tools Gateway](https://www.stackone.com/platform/agent-execution-engine/) does exactly this, reducing huge tool menus to a relevant few, shaping responses for agents, and caching results – cutting token use by up to 90% compared with naive approaches.\n\n**Data retrieval:** When an agent calls a tool, every data retrieval task involves four steps: (1) finding the right tool, (2) discovering the data, (3) figuring out how to query it, and (4) formatting the response. Most MCP servers, gateways, and coding tools handle the first and last steps well, but leave data discovery and query planning to the agent, forcing it to trawl through APIs and paginated results. [Coral](https://withcoral.com/) shifts that work to the server, letting agents send a single structured query while it handles discovery and cross-source joins behind the scenes, reducing token use by up to 64% on complex tasks.\n\n**Context avalanche:** That’s what happens when an AI keeps carrying the entire conversation forward, even though most of it is no longer relevant. After 20 rounds of edits, the model isn’t just reading your latest request, it’s also rereading the original prompt, every previous revision, tool output, and abandoned idea. That means five separate requests like “make it shorter” or “add an example” can cost much more than asking for all five changes in one go. The best fix is to batch edits where possible, and occasionally summarise the conversation and start a fresh session so the model isn’t dragging unnecessary context along with it.\n\n**Caching and compression:** Caching and compression are about not paying the model to reread the same low-value text again and again. Prompt caching makes repeated blocks like system prompts, tool definitions, and policies cheaper, as long as they stay identical; even a small change in the wrong place can break the cache. Compression tackles the other side by cutting context before it reaches the model – a short error summary instead of a 5,000-line log, or direct instructions instead of polite filler. Tools like [WOZCODE](https://www.wozcode.com/) and [Headroom](https://headroomlabs.ai/) reduce this token bloat by compressing, restructuring, or collapsing tool context before coding agents pass it to the model.\n\nIf even a company like Microsoft is [contemplating](https://www.axios.com/2026/06/16/microsoft-copilot-cowork-tokenmaxxing-cowork) using DeepSeek V4 or another open source model to reduce its Copilot Cowork token spend, it shows how serious the cost problem has become for anyone deploying agents at scale. Happily enough, we’re at a point where open source models are a viable option: their performance is improving whilst keeping costs significantly lower.\n\nAs you can see in the charts below, [z.AI](http://z.ai/)‘s new open-source AI model GLM 5.2 (designed for running long coding tasks and agentic workflows) was rated 51 on the Artificial Analysis Intelligence Index vs GPT-5.5 at 55 and Claude Fable 5 at 60. The gap between closed, frontier models built by Anthropic/OpenAI and open source models is clearly decreasing. Additionally, open source models such as DeepSeek V4 Pro are increasing model capacity [without](https://fireworks.ai/blog/deepseek-v4-pro-validating-frontier-models-for-production) correspondingly increasing inference costs, and this is evident from the Cost per Intelligence Index Task below as well, where popular open source model families like DeepSeek, GLM, MiniMax, Nemotron, Kimi and Qwen are significantly cheaper than their closed counterparts.\n\nMost people default to sending every task to the most expensive frontier model (mainly because it works, and until token bills exploded, there wasn’t any incentive to optimise). But for many use cases, it’s like lighting a candle with a flamethrower when a matchstick would suffice: ridiculously overpowered and unnecessarily expensive. In fact, [Weave](https://workweave.dev/) found that 40-60% of coding agent requests are short, structurally simple completions that an open source model like Qwen handles at parity with frontier, at roughly one-fortieth of the cost.\n\nWe see renewed urgency around using open source models controlled and operated by enterprises themselves, given what we have dubbed the “Fable Fracas,” where the U.S. government [suspended](https://www.anthropic.com/news/fable-mythos-access) all foreign nationals from accessing Anthropic’s Fable 5 and Mythos 5 models, forcing Anthropic to abruptly disable access to these models globally (Anthropic can’t reliably differentiate who is a US national and who isn’t, so everyone got unplugged). That you could lose access to your AI models overnight, without warning, is terribly alarming. Besides which, with model capabilities evolving so quickly, do you really want to commit to a single model provider when two days later a more powerful alternative emerges?\n\n“Open-source models change the sovereignty equation. Once you have the weights, your current capability is yours. That’s fundamentally different from relying on an API that can be withdrawn.”– Meryem Arik, Co-founder and CEO at Doubleword\n\nYet another lever we can pull (in addition to open source models) is to use smaller AI models for tasks that don’t require the reasoning depth or broad knowledge of frontier LLMs. Because they have lower per-token pricing and require less compute to process inputs and outputs, they can materially cut inference costs at scale: [NVIDIA Research](https://arxiv.org/pdf/2506.02153) notes that serving a 7B (7-billion-parameter) Small Language Model or SLM can be 10-30× cheaper than a 70–175B LLM, while models such as Phi-2 (which is a tiny, 2.7B model) achieved reasoning and code-generation scores on par with 30B models while running about 15× faster.\n\nThat open source and smaller models account for a significant portion of traffic in real world deployments is evident from Weave’s analysis below. We are clearly in a multi-model world, rather than an entire architecture standardised around ChatGPT/Claude.\n\nOpenRouter [found that](https://openrouter.ai/blog/announcements/fusion-beats-frontier/) combining the results of multiple models (by having a panel of models) can outperform individual models. Panels of budget models can surpass frontier models and get close to frontier panel performance – for instance, a budget panel of Gemini 3 Flash, Kimi K2.6, and DeepSeek V4 Pro beat GPT-5.5 and Opus 4.8, came within 1% of Fable 5’s score, and cost 50% as much.\n\n“The next frontier for inference is not only making models think for longer. It’s making agents run in parallel, solve the same problem from multiple angles, and converge on a better answer.”– Meryem Arik, Co-founder and CEO at Doubleword\n\nIt’s just as important to design how multiple models or agents share information, as some AI systems waste money because they pass too much information around. In a chain-style setup, each AI agent hands its full answer to the next one, so the context keeps getting bigger and bigger; in a group-chat setup, every agent sees everyone else’s messages, so the same information gets copied many times. This gets even worse when agents review each other’s work: a Writer Agent might produce a 2,000-token draft, a Reviewer Agent sends 500 tokens of feedback, and the Writer regenerates the whole thing. Without a strict stop condition, they can repeat this 5–10 times over minor issues, and because every round includes the full conversation history, one small disagreement (over formatting, of all things) can burn 50,000+ tokens fast. And we used to think petty human arguments escalated quickly.\n\nThe smarter approach is to treat context carefully: route it, compress it, and only show each model what it actually needs. Smaller specialist models can do the early work (searching, sorting, reranking, and picking the most useful information) before calling the expensive model. Glean’s Waldo is a good example: by running the smaller model first, it used c.25% fewer tokens while keeping similar success rates. Another useful trick is to give sub-agents their own separate workspaces: they can read through 50K tokens of messy information, then send back a clean 2K-token summary, so the main AI agent only sees what matters. That’s also why research like [LatentMAS](https://arxiv.org/pdf/2511.20639) is interesting: instead of forcing agents to explain every intermediate step to each other in written text, it lets them share compressed internal “thought states” directly, cutting output tokens by upwards of 70% while improving accuracy by up to 14.6%.\n\nWhen you use the APIs of closed, frontier models like Claude or ChatGPT, all the infrastructure complexity of running and serving the model to you is hidden away. That way, you don’t have to worry about latency, throughput, GPU scheduling and GPU utilisation, cold starts, failover, caching, observability, uptime and various other factors (thankfully, it’s all somebody else’s problem). But it also means that you don’t have much control, you can’t customise, and the prices are what they are. To use open source or custom models – for lower costs, stronger control or more flexibility – you have several deployment models to choose from:\n\n**Serverless inference**, where you can use open-source models via API the same way you’d use Claude/ChatGPT (no infrastructure management on your side) and you are charged $ per million tokens. Here, the inference provider absorbs the risk of idle GPUs, handles autoscaling, hides cold starts, manages model catalogues (so the provider gives you one API key and unified endpoints for accessing thousands of open-weight models, without managing model weights yourself) and charges a markup for convenience.**Dedicated endpoints**, where the customer rents GPUs or reserves capacity, and is charged $ per second. This is particularly useful for deploying your own custom models, and it gives you much more predictable performance and control – but you also take on more utilisation risk.**Self-hosting**, where you run the open model yourself, either directly on your own hardware and GPUs that you own (which exposes you to the full complexity of GPU operations) or through newer infrastructure providers (discussed later) who simplify the deployment, management, and scaling of LLMs so that self-hosting is much more straightforward.\n\nThe economics also differ sharply by workload. Serverless inference is attractive when traffic is spiky, uncertain, or early-stage, because it scales to zero and avoids idle hardware. Dedicated deployments make more sense when traffic is predictable enough to justify paying for reserved GPUs. Self-hosting becomes compelling when query volume is high, workloads are specialised, or the company can materially improve performance through custom infrastructure optimisation efforts.\n\n“We believe inference will evolve much like databases, with specialised infrastructure emerging for different workloads. Even using the same model, architecture, data and hardware, optimising inference correctly can deliver almost a 40× improvement in performance.”– Noam Salinger, Co-founder and CEO at Impala\n\nThe next generation of AI workloads raises the bar for inference infrastructure. Traditional stacks were built for interactive chat, where a human asks a question and expects an immediate response. Agentic workloads are different: they are longer-running, more asynchronous, more read-heavy, and often accumulate large amounts of context from reasoning steps, tool calls, and prior interactions. As a result, standard API-level caching is no longer enough; systems repeatedly pay prefill costs for long context windows unless they use infrastructure-level optimisations such as prefix caching, KV-cache reuse, paged attention, speculative decoding, compression, batching, and careful GPU orchestration. That’s why good inference provision isn’t just about cheaper tokens. It’s about delivering high throughput, low latency, strong uptime, predictable scaling, efficient GPU utilisation, observability, model flexibility, and workload-aware optimisation. Depending on the workload, the right architecture may be serverless, reserved capacity, or self-hosted infrastructure tuned specifically to the application.\n\nAlthough the lines are certainly more blurred than what the categorisation below suggests, here’s how we think about the landscape of providers:\n\n**Real time inference and model serving**: This category includes platforms that help developers deploy, serve, or access AI models through production-ready, low-latency endpoints. These companies are primarily oriented around interactive or user-facing inference workloads, where responsiveness, reliability, scaling, and endpoint management matter. Startups in this category include [Runware](https://runware.ai/), [Fireworks AI](https://fireworks.ai/), [Pipeshift](https://pipeshift.com/), [Cerebrium](https://cerebrium.ai/), [Baseten](https://www.baseten.co/), and [Featherless](https://featherless.ai/).\n\n**Async, batch and high-throughput inference**: This category captures inference platforms designed for high-throughput workloads that do not require an immediate response. These workloads often include background agents, evals, document processing, coding tasks, synthetic data generation, and other delayed-response AI jobs where cost and throughput matter more than latency. Providers in this space include [Doubleword](https://www.doubleword.ai/), [Sail Research](https://www.sailresearch.com/), [Impala AI](https://www.getimpala.ai/), [Sference](https://sference.com/), [Parasail](https://www.parasail.io/).\n\n**Full-stack AI compute providers**: These are broader AI infrastructure platforms that support multiple stages of the AI lifecycle, rather than only inference. They typically combine some mix of GPU access, training, fine-tuning, batch jobs, deployment, inference, orchestration, and developer tooling. Companies in this group include [Anyscale](https://www.anyscale.com/), [Together AI](https://www.together.ai/), [GMI Cloud](https://www.gmicloud.ai/en), [Modal](https://modal.com/), [Lyceum Technology](https://lyceum.technology/), and [FlexAI](https://flex.ai/).\n\n**Inference infrastructure and optimisation**: This category includes companies building the lower-level systems that make inference faster, cheaper, more reliable, or easier to run across different hardware and deployment environments. Rather than primarily selling hosted model endpoints, these companies focus on the infrastructure beneath model serving: runtimes, serving engines, GPU fleet operations, hardware abstraction, deployment tooling, and performance optimisation.\n\nThe companies in this group approach the problem from different layers of the stack. [Modular](https://www.modular.com/) and [Inferact](https://inferact.ai/) sit closest to the runtime and serving-engine layer, helping models execute more efficiently across software and hardware environments. [Kog](https://www.kog.ai/) and [TheStage AI](https://app.thestage.ai/) are more focused on inference acceleration and deployment optimisation, particularly for low-latency, performance-sensitive, multimodal, or edge use cases. [Gimlet Labs](https://gimletlabs.ai/) is more focused on heterogeneous inference infrastructure, helping AI workloads run across different types of compute and execution patterns. [Durantic](https://durantic.io/) sits closer to the infrastructure operations layer, helping teams manage GPU fleets, orchestration, serving stacks, and monitoring for production AI workloads.\n\n**Small-model inference**: Many companies increasingly want to run wide catalogues of task-specific models for text and code generation, document/audio/image processing and search, and other use cases. But traditional single-model serving is inefficient here: every model gets its own deployment and GPU pool, each provisioned for peak load and idle much of the time. A better architecture packs multiple models onto shared GPU clusters behind a unified API, improving utilisation while making it practical to self-host a broad model catalogue. That’s the problem [Superlinked](https://superlinked.com/) is solving.\n\n“Imagine any task inside a company. You could assign it either to an intern or to your principal engineer. You wouldn’t have your principal engineer perform every trivial task, and models work exactly the same way – not every task requires the highest possible intelligence.”– Adam Cohen, Co-founder and CEO at Weave\n\nAs we demonstrated earlier, AI models vary by cost and performance, so we are increasingly moving towards a multi-model world. The logical extension of this idea is that you should be able to pick and choose the lowest-cost model that’s good enough for the task, while reserving premium models for genuinely hard or uncertain tasks.\n\nThat’s exactly what routers do: they select the best model and inference provider for each request, balancing quality, cost, latency, load, and failover. Gateways extend this idea by adding the infrastructure around routing: a single API endpoint, authentication, rate limits, retries, logging, observability, cost tracking, budget controls, governance, and policy enforcement. The hard part isn’t choosing the cheapest model, but knowing when a cheaper model is good enough, which is why task-specific evals, live quality scoring, regression detection, and feedback loops are critical.\n\nThat’s how routers and gateways make multi-model systems manageable, helping teams optimise spend without sacrificing reliability. AI-native gateways include [OpenRouter](https://openrouter.ai/), [Weave](https://weaverouter.com/), [Martian](https://withmartian.com/), [Requesty](https://www.requesty.ai/), and [Bloom Factory](https://www.bloomfactory.ai/index.html). Meanwhile, established API gateways and management platforms like [Kong](https://konghq.com/) and [Tyk](https://tyk.io/) are extending enterprise traffic-management infrastructure into LLM traffic, bringing large install bases, security depth, and observability tooling.\n\n“Model routing isn’t really about switching between models. It’s about switching between vendors. You choose the right model for the task first; then you optimise for uptime, latency, and cost.”– Martin Buhr, CEO and Co-founder at Tyk\n\nWe’re also seeing applications like [Mindstone Rebe](https://www.mindstone.com/rebel)[l](https://www.mindstone.com/rebel) route not only across models, but also billing models. In practice, Rebel uses Anthropic for the higher-level reasoning and orchestration, then hands off execution to cheaper options like GPT-5.5 or open-source models. It can route those requests through OpenRouter, API providers, or even a ChatGPT Codex subscription, depending on which path is cheapest for the task or fits with the user’s subscriptions.\n\nOutput tokens cost 2–5× more than input tokens, yet most teams haemorrhage them on low-value verbosity. Left unchecked, models open with filler (“Sure!” and “Great question!”), restate the prompt, add unsolicited suggestions, and narrate their own reasoning – all billed at the premium output rate. Open source projects like [Headroom](https://github.com/headroomlabs-ai/headroom), [claude-token-efficient](https://github.com/drona23/claude-token-efficient) and [caveman](https://github.com/juliusbrussee/caveman) fix that by cutting unnecessary tokens. Somewhat amusingly in case of caveman, it gets the model to respond as a caveman (”why use many token when few token do trick” and “Brain still big. Mouth small”).\n\n## Tokenminning caveats and other key considerations\n\nWe’ve discussed the different layers of a token-efficient tech stack, from context and memory management to output optimisation. But it’s equally important to see how optimisations at these different layers interact with each other – in most cases they complement each other beautifully, but in others they don’t work so well together. For instance, this can be seen in the case of Code Mode.\n\n“Code Mode is a way for the agent to execute code in a sandbox outside of the context window to process data before it hits the model. But newer models don’t necessarily like code mode that much – they’re trained to call tools directly and aren’t great at calling tools via their code. We’ve also seen it clash with other optimisation layers: if you have code mode in the agent harness and code mode on the tools layer, it doesn’t mix very well. That’s why our current focus is simply ensuring agents receive only the precise necessary information.”\n\n*– Guillaume Lebedel, Co-founder and CTO at StackOne*\n\nWe also wouldn’t overdo context compression. Compression is excellent for removing the extra stuff you don’t need, but it’s also possible that your agent loses critical information and valuable nuance if you summarise too aggressively. The better strategy may be to use cheaper intelligence with more context than to use an expensive model with less context… that said, most open-source/small models which are cheaper also have smaller context windows, so you may have to keep context tight anyway. That’s why we prefer solutions that feed AI agents only the precise context needed at the very outset (e.g. when you use Search APIs, tool gateways, data retrieval layers) rather than cutting down unnecessary context later.\n\n“You can cut context, sure, but then you lose quality. It’s like telling an intelligent person they’re only allowed to speak to half the people they need to solve a problem.”\n\n*– Joshua Wöhle, CEO & Co-Founder at Mindstone*\n\nThus far, we have primarily focused on cost and various dimensions of performance (accuracy, latency, reliability etc.) but it is of paramount importance to ensure security and governance (particularly as we increasingly use open source models), as well as resilience and sovereignty.\n\n“As AI deployments scale from tens of agents to hundreds, cost becomes a security issue. A single runaway agent can create the AI equivalent of a denial-of-service attack, which is why the AI gateway is evolving into the enterprise control plane – not just for routing requests, but for enforcing security, governance, and cost policies.”\n\n*– Kunal Rupani, Founder and CEO at Bloom*\n\nWe’ve written about security and governance extensively over the years so we won’t belabour the point (see our work on [ Responsible AI](https://mmc.vc/research/responsible-ai/),\n\n[, and](https://mmc.vc/research/agentic-enablers-working-through-ais-insecurity-and-identity-crisis/)\n\n*Agentic Enablers: Working through AI’s insecurity and identity crisis*[). But what is interesting is that each layer of the stack is increasingly embedding security and governance features, all the way from PII detection and prompt injection prevention at tools or AI gateway layers to role-based access control at data retrieval and memory infrastructure layer. We take this as a sign of a maturing ecosystem.](https://mmc.vc/research/the-next-wave-of-agentic-security/)\n\n*The Next Wave of Agentic Security*Although we’ve spoken about Anthropic’s Fable Fracas before, it bears mentioning again because it sharply underscored the need for avoiding model vendor lock-in, and focusing on sovereign AI. Our conversations with enterprise practitioners suggests that sovereignty is increasingly a key criteria in vendor selection (provided it meets performance requirements).\n\n“I think the demand for European AI infrastructure is becoming more and more important. As the EU AI Act, GDPR and broader data governance requirements become more relevant for enterprises, we’re increasingly seeing customers come to us specifically because they’re looking for a European solution. But that only works if the product is actually competitive. Customers won’t choose a European provider just because it’s European – they’ll choose it if it’s on par with the best global providers on reliability, availability, ease of use and support. That’s what we’re trying to build.”\n\n*– Magnus Grünewald, Co-founder and CEO at Lyceum*\n\n## Return on Tokens (or the unfortunate acronym ROT)\n\nAs we explored in our [ State of Agentic AI](https://mmc.vc/research/state-of-agentic-ai-founders-edition/) research, ROI on AI spend isn’t determined purely by the quality of the underlying tech. It depends just as much on workflow design, organisational behaviour, governance and change management. The organisations extracting the highest returns aren’t necessarily those spending the least on tokens, but those converting tokens into useful business outcomes most efficiently. That’s why we think enterprises should optimise for\n\n**Return on Tokens (ROT)** rather than token cost alone.\n\nAt the very outset, it’s worth looking at what “fully loaded” token costs look like: it’s not only linked with the number of tokens your agent consumes to perform a task, but also the cost of reviewing, verifying and re-working your agent’s output. In their analysis of 30 ChatDev software engineering tasks, [Salim et al. (2026)](https://arxiv.org/pdf/2601.14470) found that iterative Code Review stage accounts for the majority of token consumption for an average of 59.4% of tokens, which suggests that the primary cost of agentic software engineering isn’t in initial code generation but in automated refinement and verification.\n\nOn top of token costs, there is also the human cost of reviewing, debugging and reworking AI-generated output: hours of engineering time multiplied by hourly rates. Faros AI [found](https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways) that while AI has accelerated code generation, bugs per developer are up 54%, code churn has surged by 861% in high-AI-adoption environments… and so median review time has increased 5x. We’ve heard similar feedback across some of our portfolio, that AI is magnifying mistakes and increasing the workload or creating bottlenecks at other stages. There are two ways to cut this: either improve accuracy at the very outset (which can be achieved by using context and memory management solutions, along with frontier intelligence or panels of open-source models that collectively approach frontier intelligence), or make the review and verification process fast and cheap. Ideally the solution is to do both.\n\nThat’s why we think token costs can’t be optimised in isolation, and it needs to be assessed in relation to outputs, using metrics such as cost per completed task, or cost per unit of useful output.\n\nWhile we’ve discussed the cost side of the ROI equation, it’s worth dwelling on the benefit side as well, and our interviews found that enterprises and startups use a wide range of north-star metrics to define AI’s value, such as:\n\n- Revenue per employee\n- Headcount reduction or no increases to headcount even as the business expands\n- Achievement of business outcomes (e.g. reduce month end close from 15 days to 3, engineering output increased by X%)\n- Time savings\n- New capabilities unlocked (so you’re able to do work you could have never done before)\n\nMeasuring ROI sounds straightforward in principle, but AI spend is unusually difficult to forecast, mainly because of how unpredictable token spend is. [Bai et al. (2026)](https://arxiv.org/pdf/2604.22750) found that frontier models fail to accurately predict their own token usage (and in fact, systematically underestimate real token costs). This is mainly because AI agents won’t know ahead of time how much context it will accumulate, how many steps it’ll take to finish, and agent trajectories are inherently probabilistic – so even if you run the same agent on the same task, the cost can vary by 30x.\n\nThat’s why we see a whole new host of tooling emerging here, whether from existing solutions across a range of categories (FinOps, developer productivity, observability) or standalone solutions like [Paid](https://paid.ai/) (which helps AI-native startups measure costs and margins whilst communicating the value their agents deliver to their customers) and [Monetise](https://monetise.dev/) (real-time economic controls for AI agents, which helps organisations enforce cost limits and prevent runaway autonomous workflows).\n\nVisibility, governance and control are only useful if organisations act on it. For instance, companies like Uber, Coinbase, Match Group, and Walmart have now [set](https://www.businessinsider.com/fintech-company-slash-employee-burned-through-thousands-in-tokens-2026-6) [caps](https://www.businessinsider.com/cfo-power-brokers-ai-era-2026-6) on how much their employees can spend on AI. However, on a startup level, we aren’t seeing clear caps per team, and it’s mainly because adoption patterns are unclear and evolving; setting tight budgets at this point may curb innovation and experimentation. For instance, one startup founder noted that 5% of employees consume roughly 40% of tokens, and this evolves as adoption grows. That’s why we’re also seeing a shift to “pooled spend” models where heavy users can tap into a pool of extra spend.\n\nYet another underrated intervention is workflow redesign. It’s worth asking whether a task genuinely requires LLM reasoning or whether it can be handled deterministically through SQL, rules or templates. Where AI is appropriate, work should be decomposed into tightly scoped steps with routing, evaluation and cost-per-outcome visibility. This reduces unnecessary token consumption, limits error propagation and ensures expensive reasoning is reserved for tasks where judgement and synthesis genuinely create value.\n\nWhile we’ve discussed technological solutions to reduce token consumption at length, it’s just one part of the puzzle. It’s equally important to design systems, workflows and governance that maximise useful work per token (and ultimately maximise Return on Tokens).\n\n## A ~~penny~~ token for your thoughts?\n\nOur position can be summarised this way:\n\n- Not everything needs an agentic solution (so it’s best to focus on workflow design)\n- Not every agent needs to use frontier intelligence (that’s why it’s worth exploring open source and small models, combined with high quality inference and routing/gateway solutions)\n- Not every agent request needs to be real time (which is why async and batch inference are interesting)\n- Not every piece of information needs to be sent to the agent (so it’s good to have context and memory management)\n\nMuch like the typical AI-native startup’s token budget today, our understanding of the token-efficient tech stack is growing by leaps and bounds. That said, we’re perpetually context hungry and keen on continuous learning, so if you’re building AI-native companies that are ROImaxxing please reach out to [Advika](https://mmc.vc/team/advika-jalan/), [Simon](https://mmc.vc/team/simon-menashy/) or [Prakriti](https://mmc.vc/team/prakriti-roy/) – we’d love to hear about it.", "url": "https://wpnews.pro/news/ai-tokenomics-how-to-tokenmin-while-roimaxxing", "canonical_source": "https://mmc.vc/research/ai-tokenomics-how-to-tokenmin-while-roimaxxing/", "published_at": "2026-07-08 07:38:20+00:00", "updated_at": "2026-07-08 08:00:36.421483+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-infrastructure", "ai-agents", "large-language-models"], "entities": ["Anthropic", "GitHub Copilot", "Uber", "ServiceNow", "Claude", "Opus 4.7"], "alternates": {"html": "https://wpnews.pro/news/ai-tokenomics-how-to-tokenmin-while-roimaxxing", "markdown": "https://wpnews.pro/news/ai-tokenomics-how-to-tokenmin-while-roimaxxing.md", "text": "https://wpnews.pro/news/ai-tokenomics-how-to-tokenmin-while-roimaxxing.txt", "jsonld": "https://wpnews.pro/news/ai-tokenomics-how-to-tokenmin-while-roimaxxing.jsonld"}}