I built AutoMode because most of my Claude Code sessions look the same. I ask a quick question, then run a real task, ask another question, then do a harder task. And by default all of it runs on Opus.
AutoMode is a free, open source (MIT) Claude Code mod. For every prompt it picks a model and a reasoning effort: Haiku for a quick fact, Sonnet for an edit, Opus for design and risky work. It runs on your own Claude login, with no API key, no proxy and no server.
The idea is obvious. Proving that it helps was not. This is the story of the benchmark, including the run where my tool lost.
I wanted numbers I'd trust if someone else had published them:
On the first 30-task run, AutoMode cost more than plain Opus.
The cause was one rule. Whenever the judge rated a task as hard with critical risk, AutoMode started it on Opus. That sent half of the 30 tasks to Opus. Plain Sonnet high passed every one of those same tasks, far more cheaply.
The fix was to make Opus something a task earns. Hard work now starts on Sonnet high. Opus comes in only on evidence: a failing check, a regression, or a reviewer blocking the change. The hardest tier still starts on Opus.
I kept the losing run in the report. If a benchmark only shows wins, I assume someone tuned it until it did.
Five mixed sessions, 20 steps. Each session runs in one continuing conversation: a question, a task, another question, a harder task.
| Setup | Passed | Total |
|---|---|---|
| AutoMode | 20/20 | $2.01 |
| Plain Sonnet high | 20/20 | $2.14 |
| effortless, on Opus | 20/20 | $3.87 |
| Plain Opus (the default) | 20/20 | $4.14 |
That is 51% less than Opus and 48% less than effortless at the same pass rate. Most of the gap is the questions. Haiku answered them for $0.08 in total, against $0.50 on Opus.
Thirty single coding tasks.
| Setup | Passed | Cost per passed task |
|---|---|---|
| AutoMode | 29/30 | $0.275 |
| Plain Opus | 28/30 | $0.499 |
| Plain Sonnet high | 30/30 | $0.240 |
The cost ratio against Opus is 0.551, with a 90% interval of 0.511 to 0.591.
No router wins every workload, so here is where AutoMode doesn't:
Model switches aren't free. Each model keeps its own prompt cache. A switch in a warm chat makes the new model write the whole conversation into its cache. AutoMode prices that rewrite before any switch, and it holds the current model when the saving wouldn't pay the rewrite back. The site has a chart where you can watch a switch pay back: https://mralexivy.github.io/AutoMode/calculator/
A risk floor in code. Migrations, auth changes, data deletion and similar work have a quality floor that the judge can't lower. Risky work always gets an independent review.
Install inside Claude Code:
/plugin marketplace add mralexivy/AutoMode
/plugin install automode@automode
/reload-plugins
The bar then shows the route for each prompt, the cache countdown and an estimated saving against the model you started on.
The rows, the method and the command to reproduce the decision are public: https://mralexivy.github.io/AutoMode/benchmark/
The code is at https://github.com/mralexivy/AutoMode. If your workload breaks my numbers, I'd like to see it. Benchmark tasks are the most useful contribution.