Multi-agent workflows: Why more agents often mean worse results

A developer who ran twenty coding tasks across three multi-agent workflow configurations found that a single-drafter system achieved a 95% score with $0.031 cost and 2.2s latency, while a planner-to-two-drafters-to-judge pipeline scored 80% at $0.692 cost and 18.3s latency—22 times more expensive and worse results. The author warns that small evaluation suites produce misleading confidence, as only three of twenty tasks showed different outcomes, and recommends measuring actual deltas before scaling agent complexity.

Multi-agent workflows: Why more agents often mean worse results The reality check I ran twenty coding tasks across three different harness shapes. The results were brutal: One drafter: 95% score | 1 call/task | $0.031 cost | 2.2s latency Planner → Drafter: 90% score | 2 calls/task | $0.264 cost | 9.1s latency Planner → Two Drafters → Judge: 80% score | 4 calls/task | $0.692 cost | 18.3s latency The complex scaffolding actually performed worse while costing 22x more. The "panel" approach didn't beat the single drafter on a single task out of twenty; in fact, it lost three. There were no errors or crashes—it just produced inferior work, slower and way more expensively. The danger of small eval suites The tool gave me a critical insight: 95% vs 80% looks decisive, but it isn't. Seventeen of those twenty tasks were ties. Only three tasks actually had different results. In statistics, three discordant tasks aren't enough to prove the panel is "worse," but they are certainly enough to prove there's no evidence the extra spend is buying anything. This is where most people mess up their AI workflow. Small test suites produce "confident nonsense." If your suite is too small, a single noise-driven failure looks like a massive regression. The only real fix is increasing the number of tasks to improve resolution. Technical breakdown of the harness To get these numbers, I treated the harness config as data—roles, models, prompts, and a topology graph. You can define the shape via JSON, declare the axes you want to vary, and run the matrix. For the scoring, I kept it strictly deterministic: 1. The system generates code. 2. Predicates execute that code. 3. A binary verdict is returned. I specifically ensured no model grades the output . This removes "judge variance" where a model might grade the same answer differently on different days. For the comparison, I used a paired per-task approach rather than comparing averages. This is an exact sign test—no assumptions about normality or variance, and ties are excluded since they provide no directional data. One quick deployment tip: if you're making browser-direct calls, watch out for CORS. api.openai.com can be tricky with access-control-allow-origin headers on the actual response, which can cause the browser to discard valid calls. If you're building an LLM agent, stop assuming more agents = better quality. Measure the actual delta before you scale the complexity. Next My Reddit-based Chair Recommendation Tool → /en/threads/3312/