Member-only story
Most AI agents don’t waste money because of the model they use. They waste money because they retrieve too much information before asking the model to think.
Everyone talks about choosing cheaper models, writing better prompts, or reducing context size to cut LLM costs.
But after working on enterprise AI agents, I noticed something else.non members can read here
The biggest source of unnecessary cost wasn’t the LLM.
It was how the agent decided what to retrieve before calling the LLM.
A smarter retrieval strategy can often save more tokens than switching to a smaller model.
A Real Example #
Imagine you’re building an AI engineering assistant connected to:
- GitHub
- Jira
- Confluence
- Logs
- Knowledge Graph
- Internal documentation
A developer asks:
“Why is the checkout service failing after yesterday’s deployment?”
A typical agent might do this:
Search GitHub↓Search Jira↓Search…