I Put My Agent's Model Routing Policy in Git. The Bill Dropped Before the Code Did.

A developer reports that an unattended coding agent spent most of its budget on a frontier model for trivial tasks like docstring regeneration and version bumps, while a subtle bug received the same expensive treatment and was later reverted. To address this, the developer implemented a model routing policy stored in a JSON file in Git, with tiered model selection based on task complexity, a verification gate, and shadow-mode validation before enabling cheaper tiers. The approach aims to make routing decisions reviewable and cost-effective.

Two weeks after I let a coding agent run unattended on backlog chores, the provider invoice told me something my logs hadn't: the agent had spent the majority of its budget on a frontier model doing work like regenerating docstrings, bumping pinned dependency versions, and normalizing import order. Meanwhile, the one genuinely subtle bug it touched — a time-ordering issue in a queue consumer — got the same model as the docstrings, with the same shallow retry behavior, and shipped a patch I reverted four days later. That inversion is the actual problem. Not "models are expensive," but: an unattended agent makes a spend decision on every single call, and the default decision is always the same. My previous post argued that agent permissions belong in version control. This is the same instinct applied to model selection: routing is policy, policy should be a file, and files should be reviewable in git. What follows is the setup I run now, with placeholders where model identifiers go — catalogs change too fast for any name I write today to survive your reading of it. Instead of burying tiers in code, the routing rules live in a JSON document that's diffable, reviewable, and rollbackable: { "tiers": { "name": "gratis", "models": "<free-tier-model-id " , "accept when": { "any of": "single-file edit", "no logic change", "docs or comments", "version bump" } }, { "name": "standard", "models": "<mid-tier-model-id " , "accept when": { "any of": "multi-file edit", "new test", "isolated function change" } }, { "name": "heavy", "models": "<top-tier-model-id " , "accept when": { "any of": "concurrency", "data migration", "public API change", "touches auth or crypto" } } , "escalation": { "attempts per tier": 1, "on exhaustion": "open issue for human" }, "gate": { "command": "make", "verify" , "timeout seconds": 900 } } Three design choices do the safety work, and none of them are about which models you pick: make verify , full stop.The runner that consumes this is uninteresting plumbing — read policy, classify task shape, call provider, run gate, escalate — maybe 120 lines. The policy file is the artifact worth keeping, because it's the thing you review, diff, and blame. The risky moment isn't writing the policy; it's trusting it. So before any task actually routes through the cheap tier, I run the classifier in shadow mode : it labels every incoming task, logs what it would have done, and the agent keeps using the strong model as usual. After a week of real traffic, I replay the log and ask: | Question from the shadow log | What "healthy" looks like | |---|---| | How often did the label say "gratis" for a task that later needed rework? | Near zero — cheap-tier mislabels are the expensive kind | | What fraction of traffic was labeled "gratis" at all? | If it's under ~40%, your tier boundaries are probably too conservative to save anything | | Did anything labeled "gratis" touch a file on my never-cheap list? | Must be zero; enforce with a path blocklist, not keyword hope | | Projected spend vs. actual spend that week | Meaningful reduction, or the whole exercise is theater | Only after the shadow numbers look sane do I let the low tier act on its own labels — and only for task shapes with a clean shadow record. The concurrency-and-auth category never graduates out of the top tier, no matter how good the log looks. One more discipline: re-run this validation whenever you swap a model in or out of a tier. The same model ID can behave differently after a provider-side update, and last month's reliable mid-tier citizen can quietly become this month's source of reverted patches. If the cheapest tier costs actual money per call, there's a real argument that routing overhead isn't worth it for small workloads. If it costs nothing , the economics flip: the expected cost of the whole system converges on the small set of tasks that genuinely need a strong model. I run this through MonkeyCode https://www.monkeycode.dev , which offers free model access and a free server option — so both the routing process itself and the bottom tier execute without a meter attached. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Two caveats I'd hold regardless of provider: I haven't verified which models sit on the free tier as you read this — pull the live model listing rather than trusting any post, this one included — and any free tier can change limits or vanish. The design above already tolerates that: a dead free model errors out of its tier and the task escalates, which is exactly what the bounded-escalation rule is for. make verify can't catch a broken patch, a cheap tier will happily pass garbage. Strengthen the suite first — that pays off even if you never route anything.If you've shadow-tested a routing policy on a real repo, I'd genuinely like to hear what your mislabel rate looked like — especially which task shapes fooled the classifier. Mine keeps getting tricked by version bumps that turn out to be breaking changes, and I haven't found a clean signal for those yet.