Does routing coding agent subtasks to cheaper models pay off? A benchmark of 63 runs per arm

A developer built Token Beggar, an MIT-licensed plugin that routes a coding agent's bulk subtasks—large reads, repeated edits, test loops—to a cheaper model with its own context while keeping design and root-cause work on the chat model. Across 63 runs per arm, total session cost fell from $9.56 to $7.69 (1.24x cheaper) with quality unchanged at 0.96, and multi-file features saw the largest gain at 1.56x. The developer cautions that routes were tuned on the same 21 tasks and that a stale .pyc file caused a false regression in one debugging task, so the scorer now clears bytecode before checking.

I wanted a straight answer to one question: if a coding agent hands its cheap subtasks to a smaller model, how much does a session really cost less, and does the result get worse? Token Beggar is a plugin that adds a routing card to the session. The chat model grades each task into one of 7 classes and delegates bulk work large reads, repeated edits, test loops to a worker with its own context, at a cheaper model and lower effort. Design and root-cause work stays with the chat model. The chat model and effort are a ceiling, so nothing is routed above what you chose. | Class | Without | With | Cheaper | |---|---|---|---| | Multi-file feature | $2.47 | $1.58 | 1.56x | | Mechanical edits | $1.11 | $0.80 | 1.4x | | Reading | $0.91 | $0.75 | 1.21x | | Design | $0.93 | $0.82 | 1.14x | | Routine code | $1.47 | $1.32 | 1.12x | | Bug root cause | $1.45 | $1.29 | 1.12x | | Review | $1.21 | $1.13 | 1.07x | | All 63 runs per arm | $9.56 | $7.69 | 1.24x | Quality was 0.96 in both arms. Design, bug root cause and review never go to a worker, yet they came out 1.07x to 1.14x cheaper. That is run to run spread plus, I suspect, a shorter path the card gives the chat model. So I only quote multipliers of 1.15x or more in the plugin's footer. The routes were tuned on the same 21 tasks, so the table describes this setup and is not a forecast for yours. One debugging task looked like a regression in 5 of 6 runs with the plugin. The diffs were identical to the passing runs. The cause was a stale .pyc: the fix changed == to = , which keeps the file size, and the bytecode written by the pre-fix test run still passed Python's mtime check. The scorer now clears bytecode before checking, and both arms were rescored. If you score code produced by agents, clear pycache first. A compaction guard that writes a handoff note before an automatic compaction and defers it up to 3 times when files were edited after the last test run. The weekly model list refresh is a GitHub Action. Repo, raw runs and the scorer: https://github.com/Arthur031221/Token-Beggar https://github.com/Arthur031221/Token-Beggar MIT .