{"slug": "routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your", "title": "Routing Down Is Easy. Knowing When Not To Is Hard: Why Cheap Models Break Your Coding Agent", "summary": "A developer building Lynkr, an open-source LLM router, explains why routing coding-agent requests to cheap local models often breaks agentic sessions. The root cause is that static routing rules measure request size rather than task stakes, so mutation-heavy tasks like file editing fail due to malformed tool calls, stale string matching, and hallucinated context. The solution involves weighting tools by risk rather than counting tokens.", "body_md": "*Disclosure: I maintain Lynkr, an open-source router whose design decisions this post explains. The failure modes described are patterns widely reported across router issue trackers and local-LLM forums — the examples are representative reconstructions, not captured transcripts. The problem is real either way; ask anyone who's routed a coding agent to a 7B model.*\n\nEveryone who gets their first LLM router working does the same thing within the hour: point the expensive coding agent at a free local model and watch the bill drop to zero.\n\nThen the agent tries to edit a file.\n\nIf you browse the issue tracker of any Claude Code router — or r/LocalLLaMA on any given week — you'll find the same story in a hundred variations. The routing works perfectly. The *session* dies anyway. The killers, in rough order of frequency:\n\n**1. Malformed tool arguments.** The agent decides to call `Edit`\n\n, and the model produces arguments that are *almost* JSON:\n\n```\n{\"file_path\": \"src/auth.js\", \"old_string\": \"if (token) {\", \"new_string\": \"if (token && !expired) {\"\n```\n\nOne missing brace. The harness rejects the call, the model retries, produces a different malformation, and you're three turns deep into fixing nothing. Frontier models emit structurally valid tool calls with boring reliability; sub-10B models do it *most* of the time — and \"most of the time,\" at 30 tool calls per session, means every session breaks.\n\n**2. Stale string matching.** `Edit`\n\n-style tools require the `old_string`\n\nto match the file exactly. Small models paraphrase from memory instead of quoting — they'll \"remember\" the line as `if (token) {`\n\nwhen the file says `if (accessToken) {`\n\n. The edit fails, the model re-reads the file, burns 2,000 tokens, tries again with a different paraphrase. This is the single most reported failure, because it *looks* like the router's fault and is actually a capability cliff.\n\n**3. Hallucinated context.** Ask a small model to run tests and it may confidently call `Bash`\n\nwith `npm test -- --grep \"auth\"`\n\nin a repo that uses pytest. It's not being stupid — it's pattern-completing from training data instead of the conversation, because instruction-following degrades faster than fluency as models shrink.\n\n**4. The infinite loop.** The subtlest one: the model calls `Read`\n\non the same file five times in a row, or greps, reads, greps the same term again. Weak models lose the thread of *what they already know* in long agentic contexts. Nothing errors — the session just stops converging while tokens burn.\n\nHere's the uncomfortable part: **none of these are the router's bug, and all of them are the router's fault.** The router made a judgment — \"this request is cheap-model-safe\" — and the judgment was wrong.\n\nMost routing setups decide with static rules: token thresholds, keyword lists, scenario slots. These fail in a specific, predictable way: **they measure the request's size, not its stakes.**\n\n\"Fix the auth bug in session.js\" is eight words. Every token-based rule on earth routes it to the small model. But those eight words unleash a read-grep-edit-test loop — the exact workload where small models faceplant. Meanwhile, \"explain the difference between optimistic and pessimistic locking, with examples\" looks expensive (long answer, technical vocabulary) and is actually *perfectly* cheap-model-safe: it's pure text generation, no tool calls, no exact string matching, nothing to break.\n\nSize and stakes are almost uncorrelated in agentic traffic. That's the whole problem.\n\nWhen I built [Lynkr](https://github.com/Fast-Editor/Lynkr)'s router, most of the design ended up being about *when not to save money*. The parts that matter:\n\n**Weight the tools, not just their count.** A request where `Grep`\n\nand `Read`\n\nare in play is research — paraphrase-tolerant, failure-tolerant, ideal for a local model. A request where `Bash`\n\n, `Write`\n\n, or `Edit`\n\nwill fire is a mutation with exact-match requirements. Lynkr assigns each tool a risk weight (`Bash`\n\n0.9, `Write`\n\n0.8, `Edit`\n\n0.7 … `Grep`\n\n0.2) and scores the request's *effective* toolset. Two requests with five tools each can land tiers apart.\n\n**Treat mid-session as a signal.** If the conversation already contains three tool results, you're inside an agentic flow with accumulated exact-state (file contents, error strings). Downgrading the model mid-flow throws away the one thing that was keeping the loop convergent. Prior tool usage and conversation depth push requests *up*-tier even when the latest message is short.\n\n**Subtract the harness baseline.** Claude Code ships ~14 tool schemas with every request — including \"hello.\" Count them naively and everything looks agentic, so nothing ever routes local and you save nothing. Score only the tools the request could plausibly use, and the safe majority routes down while the risky minority stays up.\n\n**Some patterns override everything.** Greetings and \"what does X do\" questions force-route local, always. Security-sensitive analysis force-routes to the strong tier, always — a JWT architecture question is short, toolless, and precisely the wrong place to save four cents.\n\nThe result on my own traffic: 70–90% of requests route to free local models — but they're the *right* 70–90%, which is the entire difference between \"my bill dropped\" and \"my agent broke.\"\n\n`Grep`\n\nrequest from an `Edit`\n\nrequest, it isn't routing — it's gambling on which sessions break.The router's job was never \"pick the cheapest model.\" It's \"pick the cheapest model *that won't break the session*\" — and those five extra words are where all the engineering lives.\n\nThe scorer described here is ~1,000 lines of readable Apache-2.0 JavaScript: [src/routing/complexity-analyzer.js](https://github.com/Fast-Editor/Lynkr/blob/main/src/routing/complexity-analyzer.js). Steal the design, or file an issue telling me where it misjudges — the failure cases are the interesting part.", "url": "https://wpnews.pro/news/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your", "canonical_source": "https://dev.to/lynkr/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your-coding-agent-4g33", "published_at": "2026-07-08 00:08:36+00:00", "updated_at": "2026-07-08 00:28:37.805889+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools", "ai-infrastructure"], "entities": ["Lynkr", "Claude Code", "r/LocalLLaMA", "Fast-Editor"], "alternates": {"html": "https://wpnews.pro/news/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your", "markdown": "https://wpnews.pro/news/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your.md", "text": "https://wpnews.pro/news/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your.txt", "jsonld": "https://wpnews.pro/news/routing-down-is-easy-knowing-when-not-to-is-hard-why-cheap-models-break-your.jsonld"}}