Last month, a post on r/ExperiencedDevs went viral: a company spending $1 million per month on AI API costs. Layoffs wouldn't even make a meaningful dent.
The painful part? They couldn't force teams onto cheaper models because quality genuinely dropped on complex tasks. Sound familiar?
We faced the same wall at $10K/month across our team. Here's how we solved it — and cut costs by 65% without a single developer complaint.
Most teams pick one model and use it for everything:
This is the "blanket model" trap. You're either overpaying or underperforming.
We audited 30 days of our API usage and discovered something obvious in hindsight:
| Task Type | % of Calls | Needs Frontier Model? |
|---|---|---|
| Linting & formatting | 15% | No |
| Boilerplate generation | 20% | No |
| Simple completions | 25% | No |
| Test generation | 10% | Rarely |
| Complex debugging | 15% | Yes |
| Architecture decisions | 10% | Yes |
| Code review (nuanced) | 5% | Yes |
60-70% of calls didn't need a frontier model at all. They ran identically on Haiku, Gemini Flash, or even smaller models.
But the remaining 30%? Those genuinely needed Opus-tier reasoning.
Instead of forcing a model choice at the team level, we implemented automatic routing by task complexity:
The key insight: routing should be invisible to developers. If they have to think about which model to use, they'll always pick the most powerful one (just in case). The system needs to make that decision automatically.
After 30 days of task-level routing:
The biggest surprise? Quality actually improved on some tasks. Smaller models are less prone to over-thinking simple requests. Ask Opus to format an import statement and it might refactor your entire file. Ask Haiku and it just... formats the import.
You have a few options:
Write a classifier that routes based on prompt length, keywords, or context. Crude but effective for simple cases.
def route_model(prompt, context):
if len(prompt) < 100 and context.get('task') in ['lint', 'format']:
return 'haiku'
elif context.get('task') in ['debug', 'architecture', 'review']:
return 'opus'
else:
return 'sonnet' # middle ground
Use a tiny model to classify the task before routing. Adds ~50ms latency but much more accurate.
Tools like CodeRouter handle this automatically for coding workflows — they classify by development phase (planning, implementation, testing, debugging) and route accordingly.
Start with data. Audit your actual API usage before optimizing. You'll be surprised how many calls are trivial.
Don't trust developers to self-route. They'll always pick the best model "just in case." Make it automatic.
Measure quality, not just cost. Some tasks genuinely need frontier models. Don't cheap out on the 30% that matters.
The biggest savings aren't from switching models — they're from not using expensive models when you don't need to.
I'm Bo, founder building tools for AI-powered development. If you're drowning in API costs, I've been there. Happy to chat in the comments.