{"slug": "i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code", "title": "I Put My Agent's Model Routing Policy in Git. The Bill Dropped Before the Code Did.", "summary": "A developer reports that an unattended coding agent spent most of its budget on a frontier model for trivial tasks like docstring regeneration and version bumps, while a subtle bug received the same expensive treatment and was later reverted. To address this, the developer implemented a model routing policy stored in a JSON file in Git, with tiered model selection based on task complexity, a verification gate, and shadow-mode validation before enabling cheaper tiers. The approach aims to make routing decisions reviewable and cost-effective.", "body_md": "Two weeks after I let a coding agent run unattended on backlog chores, the provider invoice told me something my logs hadn't: the agent had spent the majority of its budget on a frontier model doing work like regenerating docstrings, bumping pinned dependency versions, and normalizing import order. Meanwhile, the one genuinely subtle bug it touched — a time-ordering issue in a queue consumer — got the *same* model as the docstrings, with the same shallow retry behavior, and shipped a patch I reverted four days later.\n\nThat inversion is the actual problem. Not \"models are expensive,\" but: **an unattended agent makes a spend decision on every single call, and the default decision is always the same.**\n\nMy previous post argued that agent permissions belong in version control. This is the same instinct applied to model selection: routing is policy, policy should be a file, and files should be reviewable in git. What follows is the setup I run now, with placeholders where model identifiers go — catalogs change too fast for any name I write today to survive your reading of it.\n\nInstead of burying tiers in code, the routing rules live in a JSON document that's diffable, reviewable, and rollbackable:\n\n```\n{\n \"tiers\": [\n {\n \"name\": \"gratis\",\n \"models\": [\"\"],\n \"accept_when\": {\n \"any_of\": [\"single-file edit\", \"no logic change\", \"docs or comments\", \"version bump\"]\n }\n },\n {\n \"name\": \"standard\",\n \"models\": [\"\"],\n \"accept_when\": {\n \"any_of\": [\"multi-file edit\", \"new test\", \"isolated function change\"]\n }\n },\n {\n \"name\": \"heavy\",\n \"models\": [\"\"],\n \"accept_when\": {\n \"any_of\": [\"concurrency\", \"data migration\", \"public API change\", \"touches auth or crypto\"]\n }\n }\n ],\n \"escalation\": {\n \"attempts_per_tier\": 1,\n \"on_exhaustion\": \"open_issue_for_human\"\n },\n \"gate\": {\n \"command\": [\"make\", \"verify\"],\n \"timeout_seconds\": 900\n }\n}\n```\n\nThree design choices do the safety work, and none of them are about which models you pick:\n\n`make verify`\n\n, full stop.The runner that consumes this is uninteresting plumbing — read policy, classify task shape, call provider, run gate, escalate — maybe 120 lines. The policy file is the artifact worth keeping, because it's the thing you review, diff, and blame.\n\nThe risky moment isn't writing the policy; it's trusting it. So before any task actually routes through the cheap tier, I run the classifier in **shadow mode**: it labels every incoming task, logs what it *would* have done, and the agent keeps using the strong model as usual. After a week of real traffic, I replay the log and ask:\n\n| Question from the shadow log | What \"healthy\" looks like |\n|---|---|\n| How often did the label say \"gratis\" for a task that later needed rework? | Near zero — cheap-tier mislabels are the expensive kind |\n| What fraction of traffic was labeled \"gratis\" at all? | If it's under ~40%, your tier boundaries are probably too conservative to save anything |\n| Did anything labeled \"gratis\" touch a file on my never-cheap list? | Must be zero; enforce with a path blocklist, not keyword hope |\n| Projected spend vs. actual spend that week | Meaningful reduction, or the whole exercise is theater |\n\nOnly after the shadow numbers look sane do I let the low tier act on its own labels — and only for task shapes with a clean shadow record. The concurrency-and-auth category never graduates out of the top tier, no matter how good the log looks.\n\nOne more discipline: re-run this validation whenever you swap a model in or out of a tier. The same model ID can behave differently after a provider-side update, and last month's reliable mid-tier citizen can quietly become this month's source of reverted patches.\n\nIf the cheapest tier costs actual money per call, there's a real argument that routing overhead isn't worth it for small workloads. If it costs *nothing*, the economics flip: the expected cost of the whole system converges on the small set of tasks that genuinely need a strong model.\n\nI run this through [MonkeyCode](https://www.monkeycode.dev), which offers free model access and a free server option — so both the routing process itself and the bottom tier execute without a meter attached. *Disclosure: This article was prepared as part of MonkeyCode's product outreach.* Two caveats I'd hold regardless of provider: I haven't verified which models sit on the free tier as you read this — pull the live model listing rather than trusting any post, this one included — and any free tier can change limits or vanish. The design above already tolerates that: a dead free model errors out of its tier and the task escalates, which is exactly what the bounded-escalation rule is for.\n\n`make verify`\n\ncan't catch a broken patch, a cheap tier will happily pass garbage. Strengthen the suite first — that pays off even if you never route anything.If you've shadow-tested a routing policy on a real repo, I'd genuinely like to hear what your mislabel rate looked like — especially which task shapes fooled the classifier. Mine keeps getting tricked by version bumps that turn out to be breaking changes, and I haven't found a clean signal for those yet.", "url": "https://wpnews.pro/news/i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code", "canonical_source": "https://dev.to/hackhub_6179/i-put-my-agents-model-routing-policy-in-git-the-bill-dropped-before-the-code-did-5gga", "published_at": "2026-08-13 03:48:18+00:00", "updated_at": "2026-08-13 04:22:01.476229+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools", "mlops"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code", "markdown": "https://wpnews.pro/news/i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code.md", "text": "https://wpnews.pro/news/i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code.txt", "jsonld": "https://wpnews.pro/news/i-put-my-agent-s-model-routing-policy-in-git-the-bill-dropped-before-the-code.jsonld"}}