{"slug": "i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke", "title": "I Route Every AI Agent to the Cheapest Model with Quota. Here's What Broke", "summary": "A developer built a small Python router that assigns each AI agent task to the cheapest model with remaining quota, using a tier list (small/medium/large) that keeps model names out of role prompts, a quota_status() check that drops any model above 90% usage, and a startup-failure detector that rolls to the next candidate within 60 seconds. The router deliberately never auto-escalates a task to a larger model, exiting loudly with skip reasons instead, after cheap models produced an infinite test loop that consumed 9 GB of RAM and 419 lines of duplicated helpers. The developer reports no dollar figure for the savings.", "body_md": "I wanted coding with AI agents to be genuinely cheap, so I built a tiny Python router that sends every task to the cheapest model that still has quota left.\n\nWithin three days, a cheap model wrote a test that looped forever and ate 9 GB of RAM, another stopped halfway with \"Free usage limit reached\", and a third dumped 419 lines of duplicated helper functions into a script because it assumed \"the grader supplies tests.\" 😅\n\nEveryone is asking whether coding with AI is actually faster and cheaper. My honest answer? It's fast and nearly free, right up until a cheap model writes something that takes two hours to unwind.\n\nSo here's the router, the rules that came out of those crashes, and why it's forbidden from upgrading itself.\n\n**TL;DR:**\n\nThe first mistake I made was tying each agent prompt to a model.\n\nNow the role (\"writer\", \"designer\", \"coder\") is just a prompt with rules and skills, and **a role never names a model**. It only says what size of work it usually does.\n\nWhy does that matter? Because the moment a model is hardcoded, you can't route around outages, renamed models or an exhausted rate limit. The tier list does that for you:\n\n```\n{\n  \"small\":  [\"free-model-a\", \"free-model-b\", \"cheap-model-c\"],\n  \"medium\": [\"mid-model-a\", \"mid-model-b\"],\n  \"large\":  [\"big-model-a\", \"big-model-b\"]\n}\n```\n\nCheapest first, more capable after. Swapping a model is a one-line change, and no prompt has to know.\n\nWhen a task comes in, the router walks the requested tier and filters it:\n\n``` python\ndef select_candidates(tier, tiers, allow_paid=False, forced=None):\n    if forced:\n        return [forced], []\n    to_try, skipped = [], []\n    for cand in tiers[tier]:\n        if cand.get(\"paid\") and not allow_paid:\n            skipped.append((cand, \"paid model not allowed\"))\n            continue\n        if (q := quota_status(cand)):\n            skipped.append((cand, f\"quota: {q}\"))\n            continue\n        to_try.append(cand)\n    return to_try, skipped\n```\n\n`quota_status()` reads the CLI's local usage numbers instead of hitting a rate-limit API. If any window (rolling hours or weekly) is above **90%**, that model is out.\n\nWhy 90 and not 100? Start a big task at 96% and it tends to run dry mid-stream, leaving half-written files behind. The buffer is cheaper than the cleanup.\n\nAnd yes, you can force a model by hand. The router just warns you and gets out of the way.\n\nWhat about a model that gets picked and then falls over before doing anything?\n\nAI CLIs fail at startup in creative ways: renamed models, instant 429s, free tiers that cut you off at second zero. So the router watches the first minute:\n\n```\nSTARTUP_ERRORS = (\"429\", \"model_not_found\", \"Free usage limit\", \"quota\")\n\ndef is_startup_failure(code, seconds, log):\n    if seconds > 60:\n        return False\n    if any(e.lower() in log.lower() for e in STARTUP_ERRORS):\n        return True\n    return code != 0\n```\n\nFail fast, and it rolls to the next candidate in the same tier. Every run also leaves its prompt, raw log, final answer and the choice it made, so I can see *why* a model was skipped without guessing.\n\nDid the router make everything smooth? Not even close. A few of my favourite disasters:\n\n`model_not_found`. Lesson: only route to models you've actually run.\n(If you like this kind of thing, my [previous post on running a side project with AI agents](https://dev.to/galadev/i-run-a-side-business-with-ai-agents-heres-what-broke-in-14-days-4ch) has more.)\n\nWhen every small model fails, why not bump the task to a bigger one automatically?\n\nBecause **the router never decides that a task is bigger. Only I do.**\n\nWhen a small task fails across the board, it's rarely the model being dumb. It's usually:\n\nAuto-escalating just pays premium prices to retry a broken prompt. So when a tier runs out, the router exits loudly and lists why it skipped each model. Then I either fix the prompt or bump the size by hand.\n\nThe machine executes; the human owns the scope.\n\nI don't have a dollar figure, since most of this runs on subscriptions and free tiers rather than metered APIs.\n\nWhat it saved was **headroom and attention**:\n\nThe one rule I'd keep if I had to delete everything else:\n\n**Cheapest-with-quota is the right default for writing and small scoped edits. For anything with tests, set a higher floor.**\n\nSaving ten minutes of quota is worthless if you spend an hour reverting a cheap model's \"tests\".\n\nI use this router to keep a few small data sites running on my own, like a [cheapest fuel price map for Spain](https://fuelspot.devetty.es/?utm_source=devto&utm_medium=article&utm_campaign=cheapest-model-router&utm_content=intro) and a [public tenders finder](https://licitacionesalerta.es/?utm_source=devto&utm_medium=article&utm_campaign=cheapest-model-router&utm_content=outro).\n\nThe agents are the easy part. Getting people to find and *use* the sites is the hard one. So, for anyone who's grown a side project:\n\nI read and reply to every comment 💛\n\n*I write these with help from AI agents.*", "url": "https://wpnews.pro/news/i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke", "canonical_source": "https://dev.to/galadev/i-route-every-ai-agent-to-the-cheapest-model-with-quota-heres-what-broke-3p5h", "published_at": "2026-11-18 18:01:41+00:00", "updated_at": "2026-11-18 18:12:03.815631+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models"], "entities": [], "also_reported_by": [], "takeaway": "Keep model names out of role prompts and route through a tier list, skipping any model above 90% quota and rolling to the next candidate on startup errors within 60 seconds, but never auto-escalate tiers because that just pays premium prices to retry a broken prompt.", "recommendation": {"advice": "Route agent tasks through a cheapest-first tier list with a 90% quota cutoff and 60-second startup-failure fallback, and escalate tiers manually rather than automatically.", "playbook": "llm-cost", "playbook_url": "https://wpnews.pro/playbooks/llm-cost"}, "alternates": {"html": "https://wpnews.pro/news/i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke", "markdown": "https://wpnews.pro/news/i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke.md", "text": "https://wpnews.pro/news/i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke.txt", "jsonld": "https://wpnews.pro/news/i-route-every-ai-agent-to-the-cheapest-model-with-quota-here-s-what-broke.jsonld"}}