{"slug": "does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63", "title": "Does routing coding agent subtasks to cheaper models pay off? A benchmark of 63 runs per arm", "summary": "A developer built Token Beggar, an MIT-licensed plugin that routes a coding agent's bulk subtasks—large reads, repeated edits, test loops—to a cheaper model with its own context while keeping design and root-cause work on the chat model. Across 63 runs per arm, total session cost fell from $9.56 to $7.69 (1.24x cheaper) with quality unchanged at 0.96, and multi-file features saw the largest gain at 1.56x. The developer cautions that routes were tuned on the same 21 tasks and that a stale .pyc file caused a false regression in one debugging task, so the scorer now clears bytecode before checking.", "body_md": "I wanted a straight answer to one question: if a coding agent hands its cheap subtasks to a smaller model, how much does a session really cost less, and does the result get worse?\n\nToken Beggar is a plugin that adds a routing card to the session. The chat model grades each task into one of 7 classes and delegates bulk work (large reads, repeated edits, test loops) to a worker with its own context, at a cheaper model and lower effort. Design and root-cause work stays with the chat model. The chat model and effort are a ceiling, so nothing is routed above what you chose.\n\n| Class | Without | With | Cheaper | \n|---|---|---|---|\n| Multi-file feature | $2.47 | $1.58 | 1.56x | \n| Mechanical edits | $1.11 | $0.80 | 1.4x | \n| Reading | $0.91 | $0.75 | 1.21x | \n| Design | $0.93 | $0.82 | 1.14x | \n| Routine code | $1.47 | $1.32 | 1.12x | \n| Bug root cause | $1.45 | $1.29 | 1.12x | \n| Review | $1.21 | $1.13 | 1.07x | \n| All 63 runs per arm | $9.56 | $7.69 | 1.24x | \n\nQuality was 0.96 in both arms.\n\nDesign, bug root cause and review never go to a worker, yet they came out 1.07x to 1.14x cheaper. That is run to run spread plus, I suspect, a shorter path the card gives the chat model. So I only quote multipliers of 1.15x or more in the plugin's footer. The routes were tuned on the same 21 tasks, so the table describes this setup and is not a forecast for yours.\n\nOne debugging task looked like a regression in 5 of 6 runs with the plugin. The diffs were identical to the passing runs. The cause was a stale .pyc: the fix changed `==` to `!=`, which keeps the file size, and the bytecode written by the pre-fix test run still passed Python's mtime check. The scorer now clears bytecode before checking, and both arms were rescored. If you score code produced by agents, clear **pycache** first.\n\nA compaction guard that writes a handoff note before an automatic compaction and defers it up to 3 times when files were edited after the last test run. The weekly model list refresh is a GitHub Action.\n\nRepo, raw runs and the scorer: [https://github.com/Arthur031221/Token-Beggar](https://github.com/Arthur031221/Token-Beggar) (MIT).", "url": "https://wpnews.pro/news/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63", "canonical_source": "https://dev.to/arthur031221/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63-runs-per-arm-4cdd", "published_at": "2026-10-22 12:07:02+00:00", "updated_at": "2026-10-22 12:12:12.875596+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "developer-tools", "mlops"], "entities": ["Token Beggar", "GitHub", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63", "markdown": "https://wpnews.pro/news/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63.md", "text": "https://wpnews.pro/news/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63.txt", "jsonld": "https://wpnews.pro/news/does-routing-coding-agent-subtasks-to-cheaper-models-pay-off-a-benchmark-of-63.jsonld"}}