cd /news/ai-agents/aos-bench-can-llm-agents-manage-dyna… · home › topics › ai-agents › article
[ARTICLE · art-158470] src=arxiv.org ↗ pub= topic=ai-agents verified=true sentiment=· neutral

AOS-Bench: Can LLM Agents Manage Dynamic Workloads?

A new benchmark, AOS-Bench, shows that LLM coding agents' success on isolated tasks does not carry over to managing dynamic workloads, with every system losing 13 to 23 points on heavy workloads across five model-harness pairs. In the arXiv paper 2610.33178v1, two systems 0.6 points apart in isolated success delivered 54% and 68% of the same tasks under streaming conditions, and serial dispatch left 73% of heavy-workload tasks unclaimed. A fixed concurrency of two came within four points of the best agent-managed policy on both medium and heavy workloads, while median CPU use stayed below 30% of provisioned cores under every policy.

by read1 min views1 publishedOct 27, 2026

arXiv:2610.33178v1 Announce Type: new Abstract: Coding agents are moving from handling one request at a time to serving as persistent assistants to which users hand a stream of work: a build, a data-processing job and a bug fix may all be pending on the same machine at once. Such an agent must decide when to take on new tasks, how to delegate and overlap them, and how to stay within the machine's CPU and memory. Existing benchmarks cannot measure this, because they evaluate each task in isolation. We introduce \benchname{}, which evaluates an agent system as a workload manager: verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference. Across five model--harness pairs, isolated task success does not carry over evenly. On heavy workloads every system loses 13 to 23 points. On medium workloads, systems span 11 points in isolated success but 26 points in stream success, and two systems 0.6 points apart in isolation deliver 54% and 68% of the same tasks. Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet more elaborate management does not necessarily help. A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads, and under every policy median CPU use stays below 30% of the provisioned cores while tasks wait. These results identify workload management as a distinct capability for agents, and \benchname{} as a way to measure it.

── more in #ai-agents 4 stories · sorted by recency
── more on @aos-bench 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/aos-bench-can-llm-ag…] indexed:0 read:1min 2026-10-27 · —