Durable Coroutines for TypeScript

Restate added durable coroutines to its TypeScript SDK, giving persistent, concurrent, multi-step routines the same persistence and recovery semantics as workflows at only milliseconds of overhead per durable action rather than the 50-100 milliseconds common in workflow solutions. In production deployments, a durable step in a coroutine adds a latency overhead of P50=2.5ms and P99=9ms at rates above 10,000 durable_actions/second for a synchronously replicated 3-AZ setup on GCP, according to Restate. The feature was developed with users building durable agents, whose harnesses persist fine-grained steps such as chunks of LLM results and multiple steps per tool call, and it uses TypeScript generator functions to implement a deterministic scheduler.

Blog https://restate.dev/blog Durable Coroutines for TypeScript: Fault-tolerant Agents with millisecond overhead Igal Shilman, Stephan Ewen, Giselle van Dongen We added durable coroutines to Restate's TypeScript SDK: persistent, concurrent, multi-step routines that provide the same persistence and recovery semantics as workflows , but add only milliseconds of overhead per durable action, instead of the 50-100 milliseconds often experienced in workflow solutions<sup 1 user-content-fn-1 </sup . In production deployments, a durable step in a coroutine adds a latency overhead of P50=2.5ms, P99=9ms at rates of 10,000 durable actions/second for a synchronously replicated 3-AZ setup on GCP. We developed Durable Coroutines together with users building durable agents https://restate.dev/use-cases/replit . Modern agent harnesses persist and track their progress in fine-grained steps: chunks of LLM results, multiple steps per tool call intent, guardrail, result chunks . That granularity gives agents a stronger model when recovering from failures. But this is hard to achieve with a durable execution model that schedules and dispatches activities to workers for each persistent step. We built durable coroutines to fill that gap. An added perk of this approach is that the code can assume a durable process -like model, where durable steps can rely on local variables and make efficient use of cached resources. The coroutines implementation makes heavy use of TypeScript's generator functions to implement a deterministic scheduler controlling the execution. Durable execution originally came in the shape of workflows and activities: if you want a persistent step, you need to create an activity, which gets scheduled on some worker. In systems like Temporal, that means the workflow task cedes execution returns to the workflow orchestrator , the activity is sent to a worker to execute and return a result, and the workflow task gets executed again<sup 2 user-content-fn-2 </sup . Parallel work means setting up multiple activities, and when the workflow task cedes execution, all activities start running. This approach fits a task-queue & worker architecture. An alternative approach that Restate and other systems implemented is often described as Durable Functions https://docs.restate.dev/foundations/key-concepts . The functions can have durable steps ctx.run ... , plus durable signals, sleeps, promises , ... and compose via RPC calling other durable functions . Durable steps are faster than activities or calling other durable functions because they need to do less work: Restate maintains a streaming connection to the executing functions, sending steps and acks between the application and Restate Server. Restate's entire architecture is designed to make step commits fast https://restate.dev/blog/building-a-modern-durable-execution-engine-from-first-principles . Durable functions give developers the choice between two models: putting a unit of work that should be persisted into separate functions or activities is useful for heavier tasks, or when the work needs to be queued and dispatched to different workers/deployments and run with specific rate or concurrency limits. In contrast, programs written with durable steps & coroutines feel more like writing durable processes. Except in actual failure/recovery cases, the execution is one continuous run in the same process, letting the code reuse local resources like cached files/results or authenticated connections. Restate has always supported durable steps, but the addition of Durable Coroutines fixes the previously limited support for concurrency in that API. Both models complement each other: We often see developers put larger work like conversation turns or long-runnign tools into a separate durable function whose invocation has queueing and flow control https://restate.dev/blog/announcing-restate-1-8 and use steps/coroutines to capture the durability of substeps. Restate's Durable Coroutines https://github.com/restatedev/sdk-typescript/tree/main/packages/libs/restate-sdk-gen library takes the idea of coroutines from programming languages like Go, Rust, or Kotlin and implements a deterministic scheduler to make them compatible with journal-based durable execution. There are only a few important concepts: - Everything is a generator function because that allows Restate to tightly control the execution see How it works . If you've never worked with generator functions, you can squint your eyes and pretend that yield fulfills a similar purpose here as await does for Promises. - run ... is a single durable step, like in Restate's standard API. - spawn ... creates a new concurrently executed coroutine Task , which can have its own sequence of durable actions and can spawn further Tasks. - select , all , and other combinators combine steps and tasks in the same way as promise/future combinators do. Let's dive into an example: an agent loop that spawns tools as concurrent sub-workflows and listens for steering signals. js function agent userMessage: string : restate.Operation<string { const sandbox = yield connect ; // re-usable connection across tool calls const messages: Message = {role: "user", content: userMessage} ; const tasks = new Map<string, restate.Task<ToolResult ; // tasks in-flight let steer = restate.signal<string "steer" ; // to wait for a note from the user // Durable agent loop while true { // 1. Durable LLM call const action = yield restate.run async = llm messages ; if action.type === "final" { return action.message; } messages.push {role: "assistant", calls: action.calls} ; // 2. Spawn parallel tool calls each of them possibly a multi-step sequence for const call of action.calls { tasks.set call.id, restate.spawn performCall call, sandbox ; } messages.push { role: "tool", results: action.calls.map {id} = {id, result: "task created"} , } ; // 3. Race tool calls with new steering input const next = yield restate.select {...Object.fromEntries tasks , steer} ; if next.tag === "steer" { // process steering note messages.push {role: "user", content: yield steer} ; steer = restate.signal<string "steer" ; // renew the signal } else { // process completed tool call const result = yield tasks.get next.tag ; tasks.delete next.tag ; messages.push {role: "tool", results: result } ; } } } To see how far you can go with Durable Coroutines, have a look at Restate's agent reference architecture https://github.com/restatedev/agent , which uses them to compose steering, interruption, sync/async tools, programmatic tool calling, and more. A coding agent concurrently executes LLM calls, subagents, tools, approvals, and steering commands in a single durable concurrent process. Every step is durable, replay-stable, and recoverable. The library uses TypeScript generators instead of Promises async / await to give the SDK control over coroutine execution. With async / await , the event loop decides how the execution of different concurrent code paths interleaves. Generators instead pause at each yield until the caller explicitly resumes them. This lets the SDK decide which coroutine continues next and what result it receives. By recording the order of completion in the journal, the SDK can replay execution deterministically. Take a tool with two durable steps: a guardrail and the actual tool execution. For each step, yield pauses the coroutine until the result is there: js function runTool tool, params { const allowed = yield run = guardrail tool, params ; // true/false if allowed { return "Tool call rejected by guardrail"; } return yield run = callTool tool, params ; } // SDK drives execution via: const task = runTool tool, params ; // runs nothing yet task.next ; // starts guardrail step and pauses task.next true ; // guardrail passed: starts tool call step and pauses task.next toolResult ; // returns run doesn't return its result directly. Like all actions a call to another service, a timer, or a signal , it returns a Future . yield hands that Future to the SDK the scheduler and pauses the coroutine. The scheduler waits for the Future to complete, then calls next with the result, which becomes the value of the yield expression. The code may start additional concurrent coroutines Tasks via spawn ... . The scheduler advances all coroutines repeatedly calls next ... across yield points until each coroutine has yielded a future to a Restate action e.g., a run ... step, a signal, RPC, ... , meaning everyone is currently waiting. This is the critical point where we need to ensure that we record the order in which these actions complete, so we can deterministically replay them on recovery. The scheduler does that by running RestatePromise.race pendingActions , which automatically records the winning completion in the durable execution journal. It then resumes the waiting coroutine and starts advancing coroutines again. Exactly one coroutine executes at a time, but the I/O in their durable steps overlaps. To illustrate this, here is the execution of an agent running two concurrent tools, each consisting of a guardrail step and a tool call step. Note how the SDK runs race ... each time all coroutines are waiting on a durable step run ... . js function agent prompt { const plan = yield run = llm prompt ; const webSearch = spawn searchTool plan.query ; const weather = spawn weatherTool plan.city ; const a, b = yield all webSearch, weather ; return summarize a, b ;} next . The order of completion is recorded in the journal, for deterministic recovery. The scheduler wraps every coroutine in a fiber : the generator plus a state that is ready , waiting , or done . Ready fibers sit in a FIFO queue. The scheduler advances a fiber by calling next on its generator, which runs the coroutine's code synchronously up to its next yield . Along the way, the code can spawn additional coroutines Tasks : - run creates the durable step and returns its Future without awaiting it - spawn creates a fiber and appends it to the ready queue. The new coroutine starts only once the current routine stops running. Let's zoom in on how the scheduler moves each coroutine forward until it waits on a Restate action. Here is a simplified version, based on scheduler.ts https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/src/scheduler.ts and fiber.ts https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/src/fiber.ts : js const fibers = new Set ; // all live coroutines const ready = ; // fibers that can run async function execute handler { const main = spawn handler ; drain ; while main.isDone { // Steps are journaled in the background in the order they complete. // Race ... waits for the first completion to appear in the journal. // Replays reproduce the race deterministically from the journal. const { fiber, result } = await RestatePromise.race pendingActions ; wake fiber, result ; drain ; } return main.result; } // The Restate actions that fibers wait on function pendingActions { return ...fibers .filter fiber = fiber.waitingOn?.isRestateAction .map fiber = fiber.waitingOn.promise.map result = { fiber, result } , ; } // Advance fibers one at a time until all of them are waiting function drain { while ready.length 0 advance ready.shift ; } // Run one coroutine until it waits on something unfinished function advance fiber { let input = fiber.input; // result of what it last waited on while true { const step = fiber.generator.next input ; // runs user code if step.done { return finish fiber, step.value ; } const future = step.value; // the Future passed to yield if future.isRestateAction { // durable action: set up for next RestatePromise.race fiber.waitingOn = future; return; } if future.isDone { input = future.result; // in-memory and completed: fall through loop continue; } fiber.waitingOn = future; // in-memory and pending, put aside future.onDone result = wake fiber, result ; return; } } function wake fiber, result { fiber.waitingOn = null; fiber.input = result; ready.push fiber ; } function finish fiber, result { fibers.delete fiber ; fiber.task.complete result ; // calls the onDone callbacks } function spawn coroutine { const fiber = new Fiber coroutine ; fibers.add fiber ; ready.push fiber ; // starts once the current coroutine stops return fiber.task; } function run action { // Creates the durable step, but journals only once the scheduler awaits // the race. Directly supplies result if action is already complete replay . return restateActionFuture ctx.run action ; } At a yield , the yielded value tells the scheduler which Future the coroutine is waiting for. Because yield delegates through nested generator calls, even a run nested three generator functions deep surfaces the next time the fiber calls next . Futures represent either a Restate action or an in-memory result a Task or a channel to another one coroutine . There are basically four cases to consider: - A Restate action a step, timer, call, or signal : the fiber records it as the action it waits on, and the scheduler moves on to the next ready fiber. These actions go into the next RestatePromise.race ... call. - An in-memory Future that is already done a finished Task, or a channel that holds its value : the scheduler calls next again with the result and the coroutine keeps running. - An in-memory Future that is still pending : the fiber registers to be woken when it completes, then waits. - The end of the function : the fiber is done. The task moves every other fiber waiting on it back into the ready queue. The use of generators also gives the SDK a natural way to cancel coroutines. When a cancellation signal arrives, the SDK calls throw TerminalError on each coroutine instead of next . The TerminalError is thrown at the yield in the coroutine, and gives it a chance to run any cleanup logic in the catch or finally blocks. In-flight run closures get an aborted AbortSignal . To cancel a coroutine spawned by you, you can call task.interrupt error on the Task returned from spawn . This throws a terminal error at its next yield that cascades to the coroutines it spawned. Durable Coroutines are already running in production in various companies. For example, Replit Agent https://restate.dev/use-cases/replit is built on durable coroutines. If you like this, give it a spin and let us know what you think. Restate is open, free, and available on GitHub https://github.com/restatedev/restate . To learn more about Durable Coroutines, install the dependency: npm install @restatedev/restate-sdk @restatedev/restate-sdk-gen or get started with the template npx @restatedev/restate example typescript-hello-world-gen , and check the README https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/README.md for a quick tour. If you like what we are doing, give us a star on GitHub https://github.com/restatedev/restate , and join the community on Discord https://discord.restate.dev/ or Slack https://slack.restate.dev/ . 1. Temporal states about 50 ms of scheduling overhead per regular Activity on Temporal Cloud https://docs.temporal.io/design-patterns/local-activities and claims lower latency than self-hosted deployments https://temporal.io/blog/benchmarking-latency-temporal-cloud-vs-self-hosted-temporal . ↩ user-content-fnref-1 2. Local activities in systems like Temporal are not individually durable, only once the worflow execution yields: As a result, a sequence of two local activities doesn't persist the result of the first local activity before the second one starts. ↩ user-content-fnref-2 Keep reading Core ConceptsAIIntegrationsJune 15, 2026Giselle van Dongen https://restate.dev/blog/why-checkpointing-is-not-production-grade-durable-execution Agent checkpointing is far from production-grade resiliency Checkpointing helps with recovery, but agents are distributed applications and making them durable requires more than that. Core ConceptsMarch 11, 2026Giselle van Dongen, Francesco Guardiani https://restate.dev/blog/dealing-with-versioning-in-long-running-agents Updating AI Agents safely in production How to safely update long-running AI agents with immutable deployments and pinned executions. Core ConceptsFebruary 13, 2026Restate Team https://restate.dev/blog/a-remote-control-for-your-agents A remote control for your agents Fine-grained control over in-flight and failed requests: cancel, pause, resume, restart.