# Multi-Agent Systems: 4 Tests for When One Agent Beats Five > Source: > Published: 2026-09-22 11:35:56+00:00 A team at Anthropic built a research system where one lead agent hands work to several subagents running in parallel. On their internal research eval it beat a single agent by 90.2%. In the same write-up they said it burns about 15 times the tokens of a plain chat, and that token usage by itself explained 80% of the variance in performance on the BrowseComp benchmark. Put those two numbers next to each other and the headline changes. Five agents did not win because five heads think better than one. They won largely because the job let them spend more compute in parallel on pieces that did not need each other. That distinction decides whether a multi-agent design pays for itself or quietly fails. And there is now published data on how it fails. In 2025 a group of researchers from UC Berkeley and collaborators published [Why Do Multi-Agent LLM Systems Fail?](https://arxiv.org/abs/2503.13657). The abstract opens bluntly: the performance gains of multi-agent systems on popular benchmarks "are often minimal." So they went and looked at why. They collected more than 1,600 annotated traces across 7 popular multi-agent frameworks, labelled them with an inter-annotator agreement of 0.88 kappa, and ended up with 14 failure modes in 3 categories: Read that list again and notice what is missing. "The model was not smart enough" is not a category. The authors say it directly: failure "is not merely a function of challenges in the underlying model." Most of what goes wrong is structure. Who owns what, what counts as done, and who checks. They also tried the obvious fixes on one framework. Clearer role specifications lifted the success rate by 9.4%. Adding a high-level verification step lifted it by 15.6%. Real gains, and the authors still concluded those isolated fixes were not enough and that reliability needs deeper redesign. Here is how I read the combined evidence as an operator. Before a job gets more than one agent, it has to pass four tests. Fail any one and a single agent with a good brief will usually win on cost, speed and reliability. Anthropic's own post names the poor fit: domains where every agent needs the same context, or where there are many dependencies between agents. They add that most coding tasks have fewer truly parallelizable pieces than research does. The test is simple. Write the subtasks down. If subtask B needs the output of subtask A before it can start, you do not have parallel work. You have a pipeline, and a pipeline run by five agents is one agent with four extra handoffs, each one a chance for context to get lost. Research splits well: ten sources can be read by ten readers who never talk to each other. Refactoring a module does not. If multi-agent runs cost roughly 15 times a chat, the output has to be worth that. A question someone asks once a quarter that saves a day of reading clears the bar easily. A routine task run a hundred times a day almost never does. Price it before you build it. Take the cost of one good single-agent run, multiply by fifteen, and ask whether a person would pay that for the better answer. If the honest answer is no, the architecture is decided. System design issues were the biggest failure category, and the top modes inside it were ignoring the task specification, repeating steps and missing the termination condition. Those are all brief problems. Every agent in the system needs its own written job: the input it gets, the output it returns, the format, and the condition that means stop. "Research the competitors" is not a job. "Return the pricing page URL and the lowest paid tier for each of these six companies, as a table, and stop when all six rows are filled" is a job. If you cannot write that for every agent, you are not ready for more than one. A quarter of the failures were verification failures, and adding a verification step was the single biggest fix the researchers measured. That is the cheapest lesson in the paper. A multi-agent system without a named checker is a group chat. Someone, human or agent, has to hold the definition of correct and compare the output against it before anything ships. Not "looks good", but a check against the spec from test 3. The paid skill is moving away from choosing the model and toward designing the roles. Models will keep getting better, and the failure data says that will not fix most of this. Step repetition, missing stop conditions, one agent ignoring another, nobody verifying: those survive a model upgrade because they live in the design, not the weights. So the operators who win the next two years will think less like prompt writers and more like managers. One agent per job. A written brief per job. A clear owner for the check. And the discipline to keep a job on a single agent when it does not split, even when the five-agent diagram looks better in a demo. That also changes what you can sell. A client does not want "a multi-agent system." They want a job done on a schedule that they never have to think about. The builder who can say "this role handles your outreach, this one handles support, and this one reviews both every morning" is selling something a business already understands how to buy: staff. That is exactly how the [AI Employees](https://club.reinventing.ai/ai-employees) are built. There are eight of them, each a named role with its own job: a GTM Engineer, an SEO/AEO Employee, a Web Dev Employee, a Social Media Employee, an Ad Manager, a Sales Employee, a Customer Satisfaction Employee, and a Chief of Staff for oversight. Each one is a complete routine set for a job you would otherwise hire for, and the whole set is open source and MIT licensed on GitHub. It is the four tests above, already applied: one job per role, work that genuinely splits by function, and a role whose whole job is the check. Hire the one that matches the job you keep doing yourself and read how its brief is written. If you want to talk through where your own work splits and where it does not, come hang out in the [Vibe Coding is Life group on Facebook](https://facebook.com/groups/vibecodinglife). Bring the job you are thinking of splitting.