Member-only story
There are a hundred “10 ways to save on tokens” articles out there. This is what I actually built, on a real internal tool, and the one change that did most of the work. #
I want to skip the generic advice for a second. You’ve probably already read “shorten your prompts” and “cache your context” a dozen times. Some of it’s useful. Most of it treats token cost like a checklist instead of what it actually is: an architecture decision.
So here’s the honest version — how I approached this as a software engineer on a real internal automation tool we run, not as a list of tips copied from a blog post. The single biggest lever wasn’t prompt-shortening. It was treating “which model handles this request” as a real design decision, the same way you’d decide which service handles a request in any other backend system.
The mistake I started with (and most teams make) #
When we first wired LLM calls into this internal tool, every request — easy or hard — went to the same capable model. It worked. It was also needlessly expensive, because most of what an internal automation tool actually does isn’t hard…