{"slug": "durable-coroutines-for-typescript", "title": "Durable Coroutines for TypeScript", "summary": "Restate added durable coroutines to its TypeScript SDK, giving persistent, concurrent, multi-step routines the same persistence and recovery semantics as workflows at only milliseconds of overhead per durable action rather than the 50-100 milliseconds common in workflow solutions. In production deployments, a durable step in a coroutine adds a latency overhead of P50=2.5ms and P99=9ms at rates above 10,000 durable_actions/second for a synchronously replicated 3-AZ setup on GCP, according to Restate. The feature was developed with users building durable agents, whose harnesses persist fine-grained steps such as chunks of LLM results and multiple steps per tool call, and it uses TypeScript generator functions to implement a deterministic scheduler.", "body_md": "[Blog](https://restate.dev/blog)\n\n# Durable Coroutines for TypeScript: Fault-tolerant Agents with millisecond overhead\n\nIgal Shilman, Stephan Ewen, Giselle van Dongen\n\nWe added **durable coroutines** to Restate's TypeScript SDK: persistent, concurrent, multi-step routines that provide the **same persistence and recovery semantics as workflows**, but add only **milliseconds of overhead** per durable action, instead of the 50-100 milliseconds often experienced in workflow solutions<sup>[1](#user-content-fn-1)</sup>.\nIn production deployments, a durable step in a coroutine adds a **latency overhead of P50=2.5ms, P99=9ms at rates of >10,000 durable_actions/second** for a synchronously replicated 3-AZ setup on GCP.\n\nWe developed Durable Coroutines together with [users building **durable agents**](https://restate.dev/use-cases/replit). Modern agent harnesses persist and track their progress in fine-grained steps: chunks of LLM results, multiple steps per tool call (intent, guardrail, result chunks). That granularity gives agents a stronger model when recovering from failures. But this is hard to achieve with a durable execution model that schedules and dispatches activities to workers for each persistent step.\nWe built durable coroutines to fill that gap. An added perk of this approach is that the code can assume a *durable process*-like model, where durable steps can rely on local variables and make efficient use of cached resources.\n\nThe coroutines implementation makes heavy use of TypeScript's generator functions to implement a deterministic scheduler controlling the execution.\n\n## \n\nDurable execution originally came in the shape of workflows and activities: if you want a persistent step, you need to create an activity, which gets scheduled on some worker. In systems like Temporal, that means the workflow task cedes execution (returns to the workflow orchestrator), the activity is sent to a worker to execute and return a result, and the workflow task gets executed again<sup>[2](#user-content-fn-2)</sup>. Parallel work means setting up multiple activities, and when the workflow task cedes execution, all activities start running. This approach fits a task-queue & worker architecture.\n\nAn alternative approach that Restate and other systems implemented is often described as [Durable Functions](https://docs.restate.dev/foundations/key-concepts). The functions can have durable steps (`ctx.run(...)`, plus *durable signals, sleeps, promises*, ...) and compose via RPC (calling other durable functions). Durable steps are faster than activities (or calling other durable functions) because they need to do less work: Restate maintains a streaming connection to the executing functions, sending steps and acks between the application and Restate Server. Restate's entire architecture is [designed to make step commits fast](https://restate.dev/blog/building-a-modern-durable-execution-engine-from-first-principles).\n\nDurable functions give developers the choice between two models: putting a unit of work (that should be persisted) into separate functions or activities is useful for heavier tasks, or when the work needs to be queued and dispatched to different workers/deployments and run with specific rate or concurrency limits.\n\nIn contrast, programs written with durable steps & coroutines feel more like writing durable processes. Except in actual failure/recovery cases, the execution is one continuous run in the same process, letting the code reuse local resources like cached files/results or authenticated connections. Restate has always supported durable steps, but the addition of Durable Coroutines fixes the previously limited support for concurrency in that API.\n\nBoth models complement each other: We often see developers put larger work (like conversation turns or long-runnign tools) into a separate durable function (whose invocation has [queueing and flow control](https://restate.dev/blog/announcing-restate-1-8)) and use steps/coroutines to capture the durability of substeps.\n\n## \n\nRestate's [Durable Coroutines](https://github.com/restatedev/sdk-typescript/tree/main/packages/libs/restate-sdk-gen) library takes the idea of coroutines from programming languages like Go, Rust, or Kotlin and implements a deterministic scheduler to make them compatible with journal-based durable execution.\n\nThere are only a few important concepts:\n\n- Everything is a generator function because that allows Restate to tightly control the execution (see *How it works* ). If you've never worked with generator functions, you can squint your eyes and pretend that`yield*` fulfills a similar purpose here as`await` does for Promises.\n- `run(...)` is a single durable step, like in Restate's standard API.\n- `spawn(...)` creates a new concurrently executed coroutine (`Task` ), which can have its own sequence of durable actions and can spawn further Tasks.\n- `select()` ,`all()` , and other combinators combine steps and tasks in the same way as promise/future combinators do.\n\nLet's dive into an example: an agent loop that spawns tools as concurrent sub-workflows and listens for steering signals.\n\n``` js\nfunction* agent(userMessage: string): restate.Operation<string> {\n  const sandbox = yield* connect(); // re-usable connection across tool calls\n  const messages: Message[] = [{role: \"user\", content: userMessage}];\n\n  const tasks = new Map<string, restate.Task<ToolResult>>(); // tasks in-flight\n  let steer = restate.signal<string>(\"steer\"); // to wait for a note from the user\n\n  // Durable agent loop\n  while (true) {\n    // 1. Durable LLM call\n    const action = yield* restate.run(async () => llm(messages));\n    if (action.type === \"final\") {\n      return action.message;\n    }\n    messages.push({role: \"assistant\", calls: action.calls});\n\n    // 2. Spawn parallel tool calls (each of them possibly a multi-step sequence)\n    for (const call of action.calls) {\n      tasks.set(call.id, restate.spawn(performCall(call, sandbox)));\n    }\n    messages.push({\n      role: \"tool\",\n      results: action.calls.map(({id}) => ({id, result: \"task created\"})),\n    });\n\n    // 3. Race tool calls with new steering input\n    const next = yield* restate.select({...Object.fromEntries(tasks), steer});\n    if (next.tag === \"steer\") {\n      // process steering note\n      messages.push({role: \"user\", content: yield* steer});\n      steer = restate.signal<string>(\"steer\"); // renew the signal\n    } else {\n      // process completed tool call\n      const result = yield* tasks.get(next.tag)!;\n      tasks.delete(next.tag);\n      messages.push({role: \"tool\", results: [result]});\n    }\n  }\n}\n```\n\nTo see how far you can go with Durable Coroutines, have a look at [Restate's agent reference architecture](https://github.com/restatedev/agent), which uses them to compose steering, interruption, sync/async tools, programmatic tool calling, and more.\n\nA coding agent concurrently executes LLM calls, subagents, tools, approvals, and steering commands in a single durable concurrent process. Every step is durable, replay-stable, and recoverable.\n\n### \n\nThe library uses TypeScript generators instead of Promises (**`async`**/**` await`**) to give the SDK control over coroutine execution. With **`async`**/**` await`**, the event loop decides how the execution of different concurrent code paths interleaves. Generators instead pause at each **`yield`** until the caller explicitly resumes them. This lets the SDK decide which coroutine continues next and what result it receives. By recording the order of completion in the journal, the SDK can replay execution deterministically.\n\nTake a tool with two durable steps: a guardrail and the actual tool execution. For each step, `yield*` pauses the coroutine until the result is there:\n\n``` js\nfunction* runTool(tool, params) {\n  const allowed = yield* run(() => guardrail(tool, params)); // true/false\n  if (!allowed) {\n      return \"Tool call rejected by guardrail\";\n  }\n  return yield* run(() => callTool(tool, params));\n}\n\n// SDK drives execution via:\nconst task = runTool(tool, params); // runs nothing yet\ntask.next(); // starts guardrail step and pauses\ntask.next(true); // guardrail passed: starts tool call step and pauses\ntask.next(toolResult); // returns\n```\n\n`run()` doesn't return its result directly. Like all actions (a call to another service, a timer, or a signal), it returns a *Future*. `yield*` hands that Future to the SDK (the scheduler) and pauses the coroutine. The scheduler waits for the Future to complete, then calls `next()` with the result, which becomes the value of the `yield*` expression.\n\n### \n\nThe code may start additional concurrent coroutines (`Tasks`) via `spawn(...)`. The scheduler advances all coroutines (repeatedly calls `next(...)` across yield points) until each coroutine has yielded a future to a Restate action (e.g., a `run(...)` step, a signal, RPC, ...), meaning everyone is currently waiting.\n\nThis is the critical point where we need to ensure that we record the order in which these actions complete, so we can deterministically replay them on recovery. The scheduler does that by running `RestatePromise.race(pendingActions)`, which automatically records the winning completion in the durable execution journal. It then resumes the waiting coroutine and starts advancing coroutines again. Exactly one coroutine executes at a time, but the I/O in their durable steps overlaps.\n\nTo illustrate this, here is the execution of an agent running two concurrent tools, each consisting of a guardrail step and a tool call step. Note how the SDK runs `race(...)` each time all coroutines are waiting on a durable step (`run(...)`).\n\n``` js\nfunction* agent(prompt) { const plan = yield* run(() => llm(prompt)); const webSearch = spawn(searchTool(plan.query)); const weather = spawn(weatherTool(plan.city)); const [a, b] = yield* all([webSearch, weather]); return summarize(a, b);}\n```\n\n`next()`. The order of completion is recorded in the journal, for deterministic recovery.\n### \n\nThe scheduler wraps every coroutine in a **fiber**: the generator plus a state that is *ready*, *waiting*, or *done*. Ready fibers sit in a FIFO queue. The scheduler advances a fiber by calling `next()` on its generator, which runs the coroutine's code synchronously up to its next `yield*`. Along the way, the code can spawn additional coroutines (` Tasks`):\n\n- `run()` creates the durable step and returns its Future without awaiting it\n- `spawn()` creates a fiber and appends it to the ready queue. The new coroutine starts only once the current routine stops running.\n\nLet's zoom in on how the scheduler moves each coroutine forward until it waits on a Restate action. Here is a simplified version, based on [`scheduler.ts`](https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/src/scheduler.ts) and [`fiber.ts`](https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/src/fiber.ts):\n\n``` js\nconst fibers = new Set(); // all live coroutines\nconst ready = []; // fibers that can run\n\nasync function execute(handler) {\n  const main = spawn(handler);\n  drain();\n  while (!main.isDone()) {\n    // Steps are journaled in the background in the order they complete.\n    // Race(...) waits for the first completion to appear in the journal.\n    // Replays reproduce the race deterministically from the journal.\n    const { fiber, result } = await RestatePromise.race(pendingActions());\n    wake(fiber, result);\n    drain();\n  }\n  return main.result;\n}\n\n// The Restate actions that fibers wait on\nfunction pendingActions() {\n  return [...fibers]\n    .filter((fiber) => fiber.waitingOn?.isRestateAction)\n    .map((fiber) =>\n      fiber.waitingOn.promise.map((result) => ({ fiber, result })),\n    );\n}\n\n// Advance fibers one at a time until all of them are waiting\nfunction drain() {\n  while (ready.length > 0) advance(ready.shift());\n}\n\n// Run one coroutine until it waits on something unfinished\nfunction advance(fiber) {\n  let input = fiber.input; // result of what it last waited on\n  while (true) {\n    const step = fiber.generator.next(input); // runs user code\n    if (step.done) {\n      return finish(fiber, step.value);\n    }\n    const future = step.value; // the Future passed to yield*\n    if (future.isRestateAction) { // durable action: set up for next RestatePromise.race()\n      fiber.waitingOn = future;\n      return;\n    }\n    if (future.isDone()) {\n      input = future.result; // in-memory and completed: fall through loop\n      continue;\n    }\n    fiber.waitingOn = future; // in-memory and pending, put aside\n    future.onDone((result) => wake(fiber, result));\n    return;\n  }\n}\n\nfunction wake(fiber, result) {\n  fiber.waitingOn = null;\n  fiber.input = result;\n  ready.push(fiber);\n}\n\nfunction finish(fiber, result) {\n  fibers.delete(fiber);\n  fiber.task.complete(result); // calls the onDone callbacks\n}\n\nfunction spawn(coroutine) {\n  const fiber = new Fiber(coroutine);\n  fibers.add(fiber);\n  ready.push(fiber); // starts once the current coroutine stops\n  return fiber.task;\n}\n\nfunction run(action) {\n  // Creates the durable step, but journals only once the scheduler awaits\n  // the race. Directly supplies result if action is already complete (replay).\n  return restateActionFuture(ctx.run(action));\n}\n```\n\nAt a `yield*`, the yielded value tells the scheduler which Future the coroutine is waiting for. Because `yield*` delegates through nested generator calls, even a `run()` nested three generator functions deep surfaces the next time the fiber calls `next()`. Futures represent either a Restate action or an in-memory result (a Task or a channel to another one coroutine). There are basically four cases to consider:\n\n- **A Restate action** (a step, timer, call, or signal): the fiber records it as the action it waits on, and the scheduler moves on to the next ready fiber. These actions go into the next`RestatePromise.race(...)` call.\n- **An in-memory Future that is already done** (a finished Task, or a channel that holds its value): the scheduler calls`next()` again with the result and the coroutine keeps running.\n- **An in-memory Future that is still pending** : the fiber registers to be woken when it completes, then waits.\n- **The end of the function** : the fiber is done. The task moves every other fiber waiting on it back into the ready queue.\n\n### \n\nThe use of generators also gives the SDK a natural way to cancel coroutines.\nWhen a cancellation signal arrives, the SDK calls `throw(TerminalError)` on each coroutine instead of `next()`.\nThe TerminalError is thrown at the `yield*` in the coroutine, and gives it a chance to run any cleanup logic in the `catch` or `finally` blocks. In-flight `run` closures get an aborted `AbortSignal`.\n\nTo cancel a coroutine spawned by you, you can call `task.interrupt(error)` on the `Task` returned from `spawn`. This throws a terminal error at its next `yield*` that cascades to the coroutines it spawned.\n\n## \n\nDurable Coroutines are already running in production in various companies. For example, [Replit Agent](https://restate.dev/use-cases/replit) is built on durable coroutines. If you like this, give it a spin and let us know what you think.\n\nRestate is open, free, and available on [GitHub](https://github.com/restatedev/restate). To learn more about Durable Coroutines, install the dependency: `npm install @restatedev/restate-sdk @restatedev/restate-sdk-gen` or get started with the template `npx @restatedev/restate example typescript-hello-world-gen`, and check the [README](https://github.com/restatedev/sdk-typescript/blob/main/packages/libs/restate-sdk-gen/README.md) for a quick tour.\n\nIf you like what we are doing, give us a star on [GitHub](https://github.com/restatedev/restate), and join the community on [Discord](https://discord.restate.dev/) or [Slack](https://slack.restate.dev/).\n\n## \n\n1. \nTemporal states [about 50 ms of scheduling overhead per regular Activity on Temporal Cloud](https://docs.temporal.io/design-patterns/local-activities) and[claims lower latency than self-hosted deployments](https://temporal.io/blog/benchmarking-latency-temporal-cloud-vs-self-hosted-temporal) .[↩](#user-content-fnref-1)\n2. \nLocal activities in systems like Temporal are not individually durable, only once the worflow execution yields: As a result, a sequence of two local activities doesn't persist the result of the first local activity before the second one starts. [↩](#user-content-fnref-2)\n\n## Keep reading\n\n[Core ConceptsAIIntegrationsJune 15, 2026Giselle van Dongen](https://restate.dev/blog/why-checkpointing-is-not-production-grade-durable-execution)\n\n### Agent checkpointing is far from production-grade resiliency\n\nCheckpointing helps with recovery, but agents are distributed applications and making them durable requires more than that.\n\n[Core ConceptsMarch 11, 2026Giselle van Dongen, Francesco Guardiani](https://restate.dev/blog/dealing-with-versioning-in-long-running-agents)\n\n### Updating AI Agents safely in production\n\nHow to safely update long-running AI agents with immutable deployments and pinned executions.\n\n[Core ConceptsFebruary 13, 2026Restate Team](https://restate.dev/blog/a-remote-control-for-your-agents)\n\n### A remote control for your agents\n\nFine-grained control over in-flight and failed requests: cancel, pause, resume, restart.", "url": "https://wpnews.pro/news/durable-coroutines-for-typescript", "canonical_source": "https://restate.dev/blog/durable_coroutines", "published_at": "2026-11-04 14:42:00+00:00", "updated_at": "2026-11-04 14:51:22.823412+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure", "ai-tools"], "entities": ["Restate", "TypeScript", "Igal Shilman", "Stephan Ewen", "Giselle van Dongen", "Temporal", "GCP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/durable-coroutines-for-typescript", "markdown": "https://wpnews.pro/news/durable-coroutines-for-typescript.md", "text": "https://wpnews.pro/news/durable-coroutines-for-typescript.txt", "jsonld": "https://wpnews.pro/news/durable-coroutines-for-typescript.jsonld"}}