Agent Architecture Cost-Success Trade-offs: A Controlled Evaluation across Enterprise-Style SOPs

A controlled arXiv study (2611.12953v1) comparing four LLM agent architectures across 13 enterprise standard operating procedures found no statistically separable best design in aggregate, with task success ranging from under 10% on the hardest SOP to near 100% on the easiest. Every coordinated arm — multi-agent or skill-augmented — consumed 1.4–4.7× the tokens and 2.6–5.8× the latency of a function-calling single agent for gains that were usually small and rarely significant, making the single agent the efficient default. Implementation quality outweighed the single-versus-multi-agent choice: high-quality skills lifted success to 73.1% from 67.4%, and a targeted redesign of a multi-agent implementation became the only arm to significantly beat the baseline at 75.6% versus 67.4%.

arXiv:2611.12953v1 Announce Type: new Abstract: Enterprises increasingly use large language model agents to execute standard operating procedures SOPs : long, branching, tool-intensive workflows. Yet there is little controlled evidence for choosing among competing agentic architectures. Holding the model, tasks, and tools fixed and varying only the architecture, we compare four designs a function-calling single agent, a skill-augmented agent, centralized multi-agent orchestration, and a triage swarm; eight arms across 13 enterprise SOPs, and introduce a skill-definition quality rubric to build matched high and low-quality skill corpora. Three takeaways emerge. 1 ~\textbf{There is no universal best architecture; the right choice is SOP-dependent.} On a given SOP, task success ranges from under $10\%$ on the hardest to near $100\%$ on the easiest; no architecture is statistically separable in aggregate, and added structure only plausibly pays off on complex, branching SOPs a directional, not yet significant, trend . 2 ~\textbf{Added architecture cost must be justified by a task-success gain.} Every coordinated arm multi-agent or skill-augmented spends $1.4$--$4.7\times$ the tokens and $2.6$--$5.8\times$ the latency of the single agent for gains that are usually small and rarely significant, making the single agent the efficient default; high-quality skills do lift success to $73.1\%$ from $67.4\%$ while degraded skills fall below it. 3 ~\textbf{Implementation quality outweighs the single- versus multi-agent choice.} A plausible multi-agent implementation loses to the single agent, yet a targeted redesign of it becomes the only arm to significantly beat the baseline $75.6\%$ vs.\ $67.4\%$ .