Two releases, three mechanisms, one story. v1.113 shipped a run queue so a new run that cannot start at once gets a row and a cancel path instead of a 429. The run gets status queued, its id, its cancel registration and its first stream frames before it waits; two new events (queued and admitted) land on the stream; every start route funnels through the queue (POST /v1/runs, gRPC, MCP, batches, configured-run starts, schedules, webhooks, A2A), and v1.114 extended it to session continuations and the LLM gateway. A queued run is cancellable; the old answer is still on request with admission reject. v1.114 moved provider caps to count model calls, in a lane per model kind, so a run holds no provider slot while it runs a tool, waits for a sub-agent, or waits for a human. A single model inside a lane can be capped at one in-flight call (max_models_in_flight). And a run whose model call waits on a saturated local provider lends its global run slot to the next run in line, then takes it back ahead of every new run before the call computes. Queue-aware routing lets a run skip a busy provider for an idle candidate of its tier, once at the start and once more if it loses its place after lending. Not a priority queue, not a cross-replica queue, not a bypass for budgets.
A run that waits instead of being refused
Version 1.113 of the platform shipped a run queue that gives a new run that cannot start immediately a queued status, its id, cancel registration and first stream frames instead of a 429 rejection, with two new stream events (queued and admitted) and every start route funneling through the queue, including POST /v1/runs, gRPC, MCP, batches, configured-run starts, schedules, webhooks and A2A. Version 1.114 extended the queue to session continuations and the LLM gateway, moved provider caps to count model calls in a lane per model kind so a run holds no provider slot while running a tool or waiting for a sub-agent or human, and added queue-aware routing that lets a run skip a busy provider for an idle candidate of its tier. A single model inside a lane can be capped at one in-flight call via max_models_in_flight, and a run whose model call waits on a saturated local provider lends its global run slot to the next run in line, then reclaims it ahead of every new run before the call computes; the release states it is not a priority queue, not a cross-replica queue and not a bypass for budgets.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.