Bitdepth · Nov 18, 2026 · test run Nov 13, 2026
We ran five routine AI jobs on a top closed model and on two open models. A month of that work cost $258.53 on Claude Sonnet 5.5. It cost $11.04 on the cheapest setup that got every answer right. Four of the five jobs moved without one wrong answer. The fifth did not, and it should stay where it is.
That is a 95.7% cut. The part worth reading is which job failed, and why.
Why the bill is high #
An AI agent re-reads its whole instruction book every time it does anything. In this test that book was about 14,000 tokens, roughly 20 pages. Claude Sonnet 5.5 billed about 20,000 input tokens on each call.
At $2 per million input tokens, each call costs about 4 cents before the model answers. A check that runs every 30 minutes is 1,440 calls a month. That is $57.57 a month, mostly to hear "nothing new".
The open models charge far less for the same pages. gpt-oss-120b lists at $0.03 per million input tokens. DeepSeek V4 Pro lists at about $0.66. A call that cost 4 cents on Claude Sonnet 5.5 cost about a fifth of a cent on DeepSeek V4 Pro and about a twentieth of a cent on gpt-oss-120b.
The test #
Five made-up jobs, each with an answer that code can check:
- Status check. Every 30 minutes: is anything due? 1,440 calls a month.
- Monitoring. Every 15 minutes: is any service worth flagging? 2,880 calls a month.
- Inbox digest. Once a day: which emails need action? 150 calls a month.
- Daily briefing. Once a day. 300 calls a month.
- Ticket tagging. 50 batches of 20 tickets a day. 1,500 calls a month.
The baseline was Claude Sonnet 5.5, five runs per job. The open side was gpt-oss-120b and DeepSeek V4 Pro, ten runs per job each, on the two cheapest hosts our nightly grades allow. Temperature was zero. Code graded every answer against a stated answer. No model graded another model. The whole test billed $1.29.
What it cost, and what passed #
The rule for moving a job: the cheapest model that got every run right. The winning price for each job is in bold.
The mix costs $11.04 a month. That saves $247.49 against Claude Sonnet 5.5 on everything.
Note what the rule does. gpt-oss-120b was cheapest on the status check, but it missed one run in ten. So the status check goes to DeepSeek V4 Pro at $2.65. Cheapest is not the goal. Cheapest that passes is.
The job that should stay #
The inbox digest failed on both open models. gpt-oss-120b returned no action emails at all on 3 of 10 runs. DeepSeek V4 Pro missed on 1 of 10.
This job is different from the other four. A status check has one right answer. A digest has to judge what matters. And a missed email is the quiet kind of error: nothing breaks, so nobody notices.
The digest costs $6.06 a month on Claude Sonnet 5.5. Moving it would save under $6 and risk a missed email. Leave it.
What broke that was not the model #
Settings broke more than models did. Our first pass capped replies at 64 to 400 tokens. gpt-oss-120b returned empty text on most rows, and Claude Sonnet 5.5 was cut off mid-answer on the monitoring job. That pass measured the cap, not the job. We raised the cap to 1,500 to 2,000 tokens and ran it again. Everything above is from the second pass.
Cheap is slower. Claude Sonnet 5.5 answered in 2.4 seconds on average. gpt-oss-120b took 6.2 and DeepSeek V4 Pro took 8.1. That is fine for a job on a schedule. It is not fine if a person is waiting.
The host mattered less than we expected. We also ran hosts that our nightly grades place outside the reference. On these simple jobs one of them passed 25 of 25. The model and the settings decided the result here.
What this test does not show #
- It is a small sample. Five runs per job on Claude Sonnet 5.5, ten on each open model. Treat a pass rate as a first read.
- The jobs are made up and short. Each is one call with no tools. A real agent chains calls and carries a longer history.
- The monthly figure is one measured call times a stated schedule. Your schedule will differ.
- Caching was left at each host's default. Tuned caching would lower the Claude Sonnet 5.5 figure.
- One baseline. We have not yet run a cheap closed model as a second baseline. That is next.
What to do with this #
You can do this by hand:
- List the jobs that run on a schedule or on every incoming item, with nobody asking.
- Keep the ones whose answer is short and can be checked: a label, a yes or no, a number, a field.
- Replay the last 20 real runs of each on a cheaper model, with a generous output cap.
- Move a job only if it matches every time. Leave anything that needs judgment about what matters.
- Test again when a price or a model changes.
Or we do it for you. Send us a sample of your routine AI jobs and we send back a report: what each job costs now, which ones pass on a cheaper model, and which should stay. It is free. See a sample report.
The raw data for this test is public: data/routine/2026-11-13-pass2. Numbers against a stated answer. Not a claim about why any host answers as it does.