{"slug": "my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me", "title": "My Claude Code router cost more than Opus. Here's what the benchmark taught me", "summary": "A developer built AutoMode, a free MIT-licensed Claude Code plugin that routes each prompt to Haiku, Sonnet or Opus with a matching reasoning effort, running on the user's own Claude login without an API key or proxy. In a public benchmark, AutoMode passed 20/20 mixed-session steps for $2.01 versus $4.14 on plain Opus, and 29/30 single coding tasks at $0.275 per passed task versus $0.499 for Opus, though plain Sonnet high passed 30/30 at $0.240. The developer kept an initial losing run in the report, where a rule starting hard, critical-risk tasks on Opus made AutoMode cost more than Opus, and fixed it so Opus is only entered on evidence such as a failing check or a reviewer block.", "body_md": "I built AutoMode because most of my Claude Code sessions look the same. I ask a quick question, then run a real task, ask another question, then do a harder task. And by default all of it runs on Opus.\n\nAutoMode is a free, open source (MIT) Claude Code mod. For every prompt it picks a model and a reasoning effort: Haiku for a quick fact, Sonnet for an edit, Opus for design and risky work. It runs on your own Claude login, with no API key, no proxy and no server.\n\nThe idea is obvious. Proving that it helps was not. This is the story of the benchmark, including the run where my tool lost.\n\nI wanted numbers I'd trust if someone else had published them:\n\nOn the first 30-task run, AutoMode cost *more* than plain Opus.\n\nThe cause was one rule. Whenever the judge rated a task as hard with critical risk, AutoMode started it on Opus. That sent half of the 30 tasks to Opus. Plain Sonnet high passed every one of those same tasks, far more cheaply.\n\nThe fix was to make Opus something a task earns. Hard work now starts on Sonnet high. Opus comes in only on evidence: a failing check, a regression, or a reviewer blocking the change. The hardest tier still starts on Opus.\n\nI kept the losing run in the report. If a benchmark only shows wins, I assume someone tuned it until it did.\n\n**Five mixed sessions, 20 steps.** Each session runs in one continuing conversation: a question, a task, another question, a harder task.\n\n| Setup | Passed | Total | \n|---|---|---|\n| **AutoMode** | **20/20** | **$2.01** | \n| Plain Sonnet high | 20/20 | $2.14 | \n| effortless, on Opus | 20/20 | $3.87 | \n| Plain Opus (the default) | 20/20 | $4.14 | \n\nThat is 51% less than Opus and 48% less than effortless at the same pass rate. Most of the gap is the questions. Haiku answered them for $0.08 in total, against $0.50 on Opus.\n\n**Thirty single coding tasks.**\n\n| Setup | Passed | Cost per passed task | \n|---|---|---|\n| **AutoMode** | **29/30** | **$0.275** | \n| Plain Opus | 28/30 | $0.499 | \n| Plain Sonnet high | 30/30 | $0.240 | \n\nThe cost ratio against Opus is 0.551, with a 90% interval of 0.511 to 0.591.\n\nNo router wins every workload, so here is where AutoMode doesn't:\n\n**Model switches aren't free.** Each model keeps its own prompt cache. A switch in a warm chat makes the new model write the whole conversation into its cache. AutoMode prices that rewrite before any switch, and it holds the current model when the saving wouldn't pay the rewrite back. The site has a chart where you can watch a switch pay back: [https://mralexivy.github.io/AutoMode/calculator/](https://mralexivy.github.io/AutoMode/calculator/)\n\n**A risk floor in code.** Migrations, auth changes, data deletion and similar work have a quality floor that the judge can't lower. Risky work always gets an independent review.\n\nInstall inside Claude Code:\n\n```\n/plugin marketplace add mralexivy/AutoMode\n/plugin install automode@automode\n/reload-plugins\n```\n\nThe bar then shows the route for each prompt, the cache countdown and an estimated saving against the model you started on.\n\nThe rows, the method and the command to reproduce the decision are public: [https://mralexivy.github.io/AutoMode/benchmark/](https://mralexivy.github.io/AutoMode/benchmark/)\n\nThe code is at [https://github.com/mralexivy/AutoMode](https://github.com/mralexivy/AutoMode). If your workload breaks my numbers, I'd like to see it. Benchmark tasks are the most useful contribution.", "url": "https://wpnews.pro/news/my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me", "canonical_source": "https://dev.to/alexivy/my-claude-code-router-cost-more-than-opus-heres-what-the-benchmark-taught-me-2am2", "published_at": "2026-11-05 17:44:56+00:00", "updated_at": "2026-11-05 17:49:34.462690+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["AutoMode", "Claude Code", "Anthropic", "Opus", "Sonnet", "Haiku"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me", "markdown": "https://wpnews.pro/news/my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me.md", "text": "https://wpnews.pro/news/my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me.txt", "jsonld": "https://wpnews.pro/news/my-claude-code-router-cost-more-than-opus-here-s-what-the-benchmark-taught-me.jsonld"}}