I Built a Task Orchestrator, Then Deleted Its Best Number

A developer released ORA, a single Go binary that breaks tasks into subtasks and routes each to the cheapest capable model, then deleted a misleading cost-saving metric from its README two days before shipping. The original 68% savings claim was based on a worked example that became a measurement without real data, so the developer replaced it with actual run costs. ORA works with multiple AI agents and is available on GitHub under MIT license.

Tags: ai agents golang opensource ORA is out: a single Go binary that takes a task, breaks it into subtasks, routes each one to the cheapest model that can actually do it, runs them, and reconciles the results. It works with Claude Code, Codex, Pi, Cursor, Cline, Hermes — or standalone. go install github.com/vystartasv/ora/cmd/ora@latest ora "build a login system with JWT" ora "refactor the API to use async handlers and add tests" ora "design the database schema for a multi-tenant SaaS" --plan That's the announcement. Here's the more useful part: the README used to lead with a much better number, and I cut it two days before shipping. The original README claimed a run was 68% cheaper than sending everything to a flagship model . It looked great. It was the first thing your eye hit. It was also fake — not in the fraudulent sense, in the worse sense: it was a worked example that had quietly promoted itself to a measurement. One cheap subtask, three mid, one flagship. Cost factors of 1, 2 and 10. Seventeen units instead of fifty. Sixty-six percent, rounded up to sixty-eight somewhere along the way and never questioned again. Nobody measured anything. The diagram in the README was the evidence. First instinct: document the model, make it reproducible. I added a cost-factor table so anyone could rerun the arithmetic and land on the same figure. Then the obvious question, which I should have asked first: a flagship model doesn't just cost more per token — it does more work per token. A cheap model might take 2,000 tokens, get it wrong, and retry twice. A flagship might spend 400 and be right. A cost factor per subtask pretends those are the same event. So the metric wasn't imprecise. It was measuring the wrong thing, in a direction that flattered me. Every routing decision ORA made looked like a saving, because "saving" was defined as "didn't use the expensive one." That's not a benchmark. That's a mirror. Nothing, in the savings sense. There is no savings percentage in ORA anymore. I deleted the field from the report struct, the calculation from orchestrate.go , and the table from the README. What's there instead is smaller and true: actual run cost. Every subtask's real token usage comes back from the API. Multiply by the real per-model price. Print what the run cost. 5 subtasks · 4 models · $0.0038 No counterfactual. No imaginary flagship-only run to compare against. Just what you spent. It's a duller number, and it's strictly more useful — if you're deciding whether to adopt this thing, you want to know what it costs you, not what it theoretically saved you versus a configuration you'd never have run. Any metric that requires imagining a run that never happened is a story, not a measurement. Savings claims are counterfactual by construction. So are most "X% faster" and "Y% cheaper" numbers in AI tooling right now — including a lot of the ones you've read this month. They compare a thing that happened against a thing someone assumed would have happened, and the assumption is always chosen by the person with the announcement to make. Actual cost, actual tokens, actual latency: those you can just print. The bar for a number in your README should be that a stranger can reproduce it without believing anything you say. Now that there's nothing to oversell: .ora-report.json Drop ORA.md into any agent that reads a rules file and it learns the same workflow — CLAUDE.md , .cursor/rules/ , .clinerules/ , Copilot instructions, Hermes skills. Routing is the whole idea. Not every subtask needs the best model in the world, and the routing decision is one you'd otherwise make by hand, badly, every time. GitHub: vystartasv/ora https://github.com/vystartasv/ora — MIT, single Go binary, v0.1.0. Tolerance, resilience, or just tired? Neither — this one was caught in time. Barely.