{"slug": "multi-agent-coordination-benchmark", "title": "Multi Agent Coordination Benchmark", "summary": "A new benchmark called MAC Bench, built from twelve natively multi-agent tasks spanning air traffic control to Mars sample-return mission design, finds that frontier models from OpenAI, Anthropic and Google coordinate poorly in teams: a team of eight agents scores 0.42 on average versus 0.35 for a single agent, while eight agents without coordination score 0.44, indicating the team's advantage comes from extra agents rather than communication. Only Anthropic's Opus 5.5 teams gained from messaging, and Opus 5.5 reached 0.53 in teams of eight versus 0.41 for GPT-6.1 Sol despite similar single-agent scores, while a team of eight costs about nine times as much as one agent and reaches the single agent's score only about twice as fast. The benchmark isolates the performance gain attributable to coordination alone by evaluating each task in single-agent and multi-agent settings with and without coordination.", "body_md": "## Introduction\n\nIn September 2026, OpenAI deployed approximately 10,000 agents to solve the Navier-Stokes existence and smoothness problem, arriving at a solution in just 88 hours. Scaling the number of agents is emerging as a new axis for scaling test-time compute ([Park et al., 2026](https://arxiv.org/abs/2609.21032)). But more agents do not necessarily mean better results. Agents must coordinate effectively to make progress, and poor coordination can undermine the benefits of scaling. When Anthropic used teams of agents to formalize Fermat's Last Theorem, early attempts failed because agents \"quickly lost track of the project's state and stopped collaborating effectively\" ([Anthropic, 2026](https://www.anthropic.com/research/formalizing-fermats-last-theorem)). While existing benchmarks measure how well agents complete tasks, they do not directly isolate the contribution of coordination to a team's performance ([DeepMind, 2026](https://x.com/a16z/status/2122401706340429962)). How do we measure whether agents are actually getting better at working together?\n\nMulti Agent Coordination Bench (MAC Bench) is a benchmark of twelve tasks that measures how well teams of agents coordinate. The tasks are natively multi-agent by construction: each one is built so that the work of one agent depends on the work of its teammates, in domains that range from air traffic control to the design of a Mars sample-return mission. We evaluate each task in a single-agent setting and in a multi-agent setting with and without coordination, and isolate the gain in performance that is due to coordination alone.\n\nOur main finding is that current frontier agents do not coordinate well in teams. We evaluate models from OpenAI, Anthropic and Google and find:\n\n- **Models coordinate poorly.** Across the three models, a team of eight scores 0.42 on average and a single agent 0.35 ([Figure 5](#fig-overtime) ). In the no-coordination setting, eight agents score 0.44, so the team's advantage comes from the extra agents, not from their communication. Only Opus 5.5 teams gain anything from messaging (see[the leaderboard](#fig-scores) ).\n- **Models with similar single-agent scores perform very differently in a team setting.** Opus 5.5 and GPT-6.1 Sol score about the same alone, but in teams of eight Opus 5.5 reaches 0.53 and GPT-6.1 Sol 0.41 (see[the leaderboard](#fig-scores) ).\n- **Larger teams get increasingly expensive and inefficient at problem solving.** A team of eight costs about nine times as much as one agent ([Figure 12](#fig-cost) ) and reaches the single agent's score only about twice as fast ([Figure 13](#fig-time) ).\n\nThese results leave headroom at three levels: score, compute and coordination. MAC Bench targets this headroom directly. Its tasks are constructed so that the work of each agent depends on the work of its teammates, and its settings separate the gain from coordination from the gain from adding more agents.\n\n## Benchmark Construction\n\nThe benchmark comprises twelve tasks in simulated environments. Each task is constructed for a team and requires coordination, because the work of one agent depends on that of its teammates. The [Tasks tab](#tasks) lists them.\n\nMulti-agent benchmarks such as CooperBench and TeamBench ([Kim et al., 2026](https://arxiv.org/abs/2605.07073)) mostly use short tasks. Long-horizon benchmarks such as ProgramBench ([Yang et al., 2026](https://arxiv.org/abs/2605.03546)) and FrontierCS 2 ([Mang et al., 2026](https://arxiv.org/abs/2610.29932)) are used to evaluate a single agent. Their tasks can also be run with several agents, but each task was written for one agent, so its construction does not require the agents to coordinate, and the benchmark does not measure a coordination gap. To our knowledge, no earlier benchmark combines tasks that take hours with a single-agent baseline and a setting without coordination.\n\nWe categorize a task by two properties of its environment, and we take both from the definition of a multi-agent environment. In the standard formal model, the partially observable stochastic game, all agents act in one world, which has one state and one rule for how that state changes. Each agent then has three things of its own: what it observes, what actions it can take, and what reward it receives. The world is common to all agents by definition, so these three are the only places in which the agents of one environment can differ. Our first property asks whether the agents differ in what they observe and can do, and the second asks whether they differ in what they are rewarded for. Every multi-agent task answers both questions, so every task has exactly one type.\n\nThe first property is whether the environment divides the inputs and outputs of the agents. In a *shared* task, all agents start with the same information and the same actions, and a single agent could complete the task alone given sufficient time. In a *distributed* task, the environment gives agents different information, different actions or both: each holds its own position in the system or sees only part of it, so that no agent observes or controls the whole system. The test is whether swapping two agents at the start would change what either of them can see or do. In Power Outage it would, because only the holder of a control area can read its data, although any agent may write any area's controller. In Mindustry it would too: every agent sees the whole map, but each controls only its own units. In Dubai ATC it would not: every agent sees the whole radar and can open any controller position, and the team decides who works which. In Mars Return it would not either. Only the agent that orders a test sees its result, but any agent can order any test, so the agents start alike. A task counts as distributed only if its environment forces the division. How a team chooses to organize its work is part of its solution and does not enter the categorization.\n\nThe second property is whether the agents pursue the same goal. With the same goal, every agent is scored on the same outcome. With different goals, agents are scored on conflicting outcomes. A task in which teams compete has different goals, even though the agents within one team share a goal. The two properties give four types of task. In *shared* and *distributed* tasks, all agents pursue the same goal. *Commons* and *strategic* tasks divide the environment in the same two ways, and their agents pursue different goals.\n\nEach property decides what kind of coordination a task demands. Where inputs and outputs are shared, any agent could do the work of any other, so coordination means dividing the work and keeping agents from repeating or undoing each other's work. Where they are split, an agent depends on what only its teammates can see or do, so findings and requests have to pass between agents. Where goals are the same, an agent can take a teammate's message at its word. Where goals differ, an agent has to weigh what others say against what they want. Other properties, such as whether an environment runs on its own clock, how long a task takes or how hard it is, apply to single-agent environments as well. They describe the environment and not the relation between its agents, so they do not enter the categorization.\n\nSharedExampleMars Return\n\nThe team designs a Mars sample-return mission and writes its flight software in one repository.\n\nDistributedExampleHoldout-game\n\nEach agent controls one survivor in a game that never pauses, and sees only the tiles near it.\n\nCommonsNot in MAC Bench v1Rivals in one system\n\nAgents with conflicting goals act on one shared system, for example firms that trade in one market.\n\nStrategicNot in MAC Bench v1Rivals in separate positions\n\nEach side holds its own part of the system and pursues its own goal, for example kingdoms at war.\n\n- agent\n- agent with a different goal\n- environment\n\nMAC Bench measures how well agents collaborate, so it contains only tasks in which all agents pursue the same goal: shared and distributed tasks. Commons and strategic tasks, whose agents pursue different goals, do not fit this measurement, for three reasons. First, the environment of an agent is no longer fixed. Its opponents are part of what it faces, so models that play against one another each face a different task. Second, scores and rankings depend on which models take part, and they can change when a model is added to the benchmark. Third, the coordination gain cannot be measured cleanly. Its baseline, the same agents without coordination, would silence every side at once, so the gain of one model could not be separated from the change in its opponents. Holding the opponents fixed, for example by playing every model against the same team of one model, removes these problems. That team then becomes part of the environment, and the task becomes a shared or a distributed one.\n\n### Shared Tasks\n\nNine of the twelve tasks are shared: Mars Return, SOTA Compressor, 0 A.D., Seed to Profit, Chicago Skyline Survey, RNA Secondary Structure, Spectrum to Molecule, Cave Rescue and Dubai ATC. In seven of them the team builds software in one repository. In Dubai ATC the agents control the traffic of one airport, and each opens the controller positions it chooses to work. In Cave Rescue the agents command one fleet of drones.\n\n**Example: Mars Return.** The team designs, builds and flies a Mars sample-return mission: launch from Kennedy, land at Jezero crater, spend 30 sols collecting 30 sample tubes, launch from Mars and land the samples in Utah. In one repository, the agents describe the whole vehicle in SysML v2, a systems engineering language, and write the flight software that flies it. A program office builds what they describe and runs the tests they order, such as hot fires, structural tests, wind tunnel runs and parachute drops, under a hardware budget of $1.6 billion and an R&D budget of $240 million. The grader flies the committed design through 48 hidden worlds with different atmospheres, winds, dust storms and hardware variations, and scores how far each flight gets along the mission. A replay of one of its flights and a timeline of one team are in [Figure 1](#fig-marsreturn).\n\n**Why it is shared.** Every agent has the same repository, the same tools and the same access to the program office, so any agent could do any part of the work, and in the single-agent runs one agent does all of it. The instruction names seven parts into which the mission splits naturally, from the launcher to Earth entry and recovery, but it leaves the split to the team. One rule looks like a division and is not one: only the agent that orders a test sees its result. Any agent can order any test, so the rule gives no agent a capability that another lacks. It means only that a result reaches the team when the agent that ordered it shares it.\n\n**Coordination requirement.** The parts have to fit together exactly, and each part changes the others: a heavier lander needs a bigger aeroshell and a heavier launcher, and a different launch date changes everything downstream. The two budgets and the test facilities are shared, so money or test time that one agent spends is no longer available to the others. A test result holds only for the exact design of the part tested, so a teammate's later change to that part makes the test worthless. Coordination here means owning parts, agreeing on the interfaces between them and deciding who spends the budget.\n\n**What we learn.** Averaged over the three models, a team of eight scores 0.26 in Mars Return. It is above a single agent (0.21) and below the best of eight single agents (0.38). More agents help only up to a point: Opus 5.5 scores 0.57 with a team of four and 0.33 with a team of eight. In a shared task, each added agent is one more writer to the same files and one more claim on the same budgets, and beyond some team size that cost outweighs the extra work.\n\n### Distributed Tasks\n\nThree tasks are distributed: Holdout-game, Mindustry and Power Outage. In each, the environment gives the agents different information, different actions or both.\n\n**Example: Holdout-game.** Each agent plays one survivor in a zombie survival game on a map of 96 by 96 tiles. The world never pauses, and the 4 hours of a run are 16 game days. The team builds a stronghold up a ladder of 12 buildings, from a campfire to a radio mast and a helipad. The score is the weighted share of the ladder that the team has built, and a rescue by helicopter scores 1.0. From the third night a horde attacks the walls every night, and it grows from 60 zombies to 210. Fourteen strangers ask to join, and some of them hide a bite, steal or fake a trade. [Figure 2](#fig-holdout) shows a team of eight on one night of the game.", "url": "https://wpnews.pro/news/multi-agent-coordination-benchmark", "canonical_source": "https://benchmarks.bespokelabs.ai/multi-agent-coordination-bench/", "published_at": "2026-11-18 18:28:05+00:00", "updated_at": "2026-11-18 18:45:23.209643+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "artificial-intelligence"], "entities": ["OpenAI", "Anthropic", "Google", "MAC Bench", "Opus 5.5", "GPT-6.1 Sol", "DeepMind", "Park et al."], "also_reported_by": [], "takeaway": "On MAC Bench's twelve coordination-dependent tasks, eight-agent teams average 0.42 versus 0.44 for eight uncoordinated agents and 0.35 for a single agent, so added agents rather than communication drive team gains, and a team of eight costs about nine times a single agent while reaching its score on", "recommendation": {"advice": "Benchmark your agent teams against a no-coordination baseline before scaling headcount; MAC Bench found eight-agent teams scored 0.42 versus 0.44 uncoordinated, so extra agents, not messaging, drove the gain.", "playbook": "agent-architecture", "playbook_url": "https://wpnews.pro/playbooks/agent-architecture"}, "alternates": {"html": "https://wpnews.pro/news/multi-agent-coordination-benchmark", "markdown": "https://wpnews.pro/news/multi-agent-coordination-benchmark.md", "text": "https://wpnews.pro/news/multi-agent-coordination-benchmark.txt", "jsonld": "https://wpnews.pro/news/multi-agent-coordination-benchmark.jsonld"}}