{"slug": "decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm", "title": "Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows", "summary": "A tail-risk-aware turn release scheduling method for agentic LLM workflows reduces the P95 of workflow flow time under contention by up to 3.50x, according to an arXiv paper (2609.10964v1). The method, which uses a mean-Conditional Value-at-Risk (CVaR) objective and adapts the released work budget to observed queue pressure, performs comparably to eager release under light load. It was evaluated on real agent execution traces from software engineering tasks across multiple LLMs and workflow arrival rates.", "body_md": "arXiv:2609.10964v1 Announce Type: new \nAbstract: Agentic LLM workflows consist of sequences of model turns interleaved with tool interactions, so their end-to-end completion time depends not only on inference speed but also on when ready turns are released. Most runtimes release each turn immediately upon readiness. Under contention, this eager release policy can accumulate released but unfinished work; once submitted, those turns can no longer be reordered by the workflow-level policy, increasing tail latency. We present a tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next and how much released but unfinished work to maintain. The method uses a mean--Conditional Value-at-Risk (CVaR) objective to capture the evolving tail risk of unfinished workflows, incorporates online estimates of turn work when prioritizing ready turns, and adapts the released work budget to observed queue pressure. We evaluate the method using real agent execution traces from software engineering tasks across multiple LLMs and workflow arrival rates. The method performs comparably to eager release under light load and substantially reduces the P95 of workflow flow time under contention, achieving up to a $3.50\\times$ speedup.", "url": "https://wpnews.pro/news/decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm", "canonical_source": "https://arxiv.org/abs/2609.10964", "published_at": "2026-09-12 04:00:00+00:00", "updated_at": "2026-09-12 04:26:51.859844+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-research", "ai-infrastructure"], "entities": ["arXiv", "Conditional Value-at-Risk"], "alternates": {"html": "https://wpnews.pro/news/decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm", "markdown": "https://wpnews.pro/news/decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm.md", "text": "https://wpnews.pro/news/decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm.txt", "jsonld": "https://wpnews.pro/news/decoupling-readiness-from-release-for-tail-aware-scheduling-of-agentic-llm.jsonld"}}