AI Transport v0.5.0 is now available. It adds first-class support for running an agent turn inside a durable execution framework, such as Temporal or Vercel's Workflow Development Kit (WDK), while every client watching the conversation still sees one clean, resumable stream.
The last release, v0.4.0, let an agent hydrate its history from your own database. This one is about what happens when the process running the agent isn't around for the whole turn.
Two kinds of durability #
A durable execution framework solves one half of running an agent reliably. Temporal and WDK break an agent turn into retryable units of work: an LLM call, a tool execution, a follow-up call. Each unit runs in its own process, and the framework retries the ones that fail. If a process crashes, a deploy cycles the container, or a spot instance gets reclaimed halfway through, the framework replays from the last unit that completed.
That leaves the other half: the conversation the user is watching. The agent's output has to reach the browser, survive a dropped connection, and stay in sync across the laptop and the phone they left the tab open on.
Temporal shipped Workflow Streams and WDK has resumable streams, so the agent's output is durable right up to the edge of the framework. The gap is the step from that edge to the devices a person is watching from. Temporal's Workflow Streams emit to a Temporal SDK client, not to a browser, so you still run a web server that subscribes to the stream and relays it over SSE or WebSocket, and you own the reconnect: each client has to track the offset it resumes from. A Vercel WDK stream does reach the browser, but as a single HTTP response replayed from a position, so a second tab opens its own connection with its own cursor rather than joining one shared view. Getting a single agent turn to every device on the conversation, holding it open across a reconnect, and doing that without hand-rolling per-client offsets, is still your problem.
Delivering an agent's output to clients reliably and at scale is what AI Transport is for. It carries the conversation over an Ably channel, so the stream persists on the channel, a reconnecting or late-joining client catches up from history, and every tab and device subscribed to the channel sees the same turn as it streams. v0.5.0 makes that transport safe to drive from a framework that has automatic retries.
The problem retries create #
When a durable framework retries a unit of work, it runs your code again. If that code already published output to the user on the first attempt, a plain retry publishes it a second time, and the user sees the reply twice. Keeping external side effects idempotent under retry is left to you; Temporal's own guidance is to build an idempotency key from the run and activity ids and dedupe against it.
Publishing to a live conversation is exactly that kind of side effect. v0.5.0 handles it with Steps.
Steps, the re-attemptable unit of output #
A Step is a bracket around one publishable unit of a run's output. You open it, stream through it, and close it, and it carries a stable stepId
:
const step = run.createStep({ stepId });
await step.start();
await step.pipe(llmStream);
await step.end();
The stepId
is what makes a retry safe. Two ai-step-start
events under the same stepId
coalesce on the channel: the later attempt supersedes the earlier one's output instead of appending beside it. A retried unit of work re-emits the same step, its output replaces the dead attempt's, and the reply settles once. No duplicate bubble, no orphaned half-stream.
run.pipe()
still works as before and now brackets its output in an implicit step, so a single-process agent needs no changes. You reach for createStep
with an explicit stepId
when a retry might re-run the same work in a different process.
Adopt an open run from a fresh process #
A run's lifecycle can now span several processes under one stable runId
. The process that opens the run calls run.start()
as usual. A later process, a retried activity or a follow-up invocation, adopts the run instead of opening a new one:
const run = session.adoptRun({ runId, invocationId, triggerEventId });
await run.load(); // resolves the open run from the channel; publishes no new run-start
const step = run.createStep({ stepId });
await step.start();
await step.pipe(llmStream);
await step.end();
load()
waits for the run's ai-run-start
to be observed on the channel and adopts it for publishing without republishing an opening event, so clients never see the turn restart. Combined with a stable stepId
, a fresh process can pick up an in-flight turn and continue it as if it had been there the whole time.
Run it on Temporal #
Temporal reruns a failed activity under the same activityId
. The new @ably/ai-transport/temporal
entry point ships stepIdFor
, which composes the run's invocation id with the current Temporal activityId
:
import { stepIdFor } from '@ably/ai-transport/temporal';
const run = session.adoptRun({ runId, invocationId, triggerEventId });
await run.load();
const step = run.createStep({ stepId: stepIdFor(invocationId) });
Because the activityId
is identical on a retry, the derived stepId
matches, and the retried activity's output supersedes the failed attempt's on the channel. Each temporal activity is a single SDK Step. The runnable Temporal demo has a deliberately flaky tool so you can watch a retried activity working with Steps in the Temporal Web UI and in the chat at the same time.
When running on Temporal, use stepIdFor(invocationId)
to get unique identifiers in the format ${invocationId}-${activityId}
so that Temporal can safely re-use an activityId
across different workflow runs without those Steps colliding on the Ably channel. A Temporal activityId
is guaranteed to be unique within a single workflow run only, the composite of invocationId
and activityId
makes them safe to use for Steps.
Run it on Vercel Workflows #
WDK gives each step an id that is stable across retries, so it passes straight into createStep
with no composition:
import { getStepMetadata } from 'workflow';
const { stepId } = getStepMetadata(); // stable across retries of this activity
const run = session.adoptRun({ runId, invocationId, triggerEventId });
await run.load();
const step = run.createStep({ stepId });
There's no separate worker or account to run: withWorkflow
compiles the workflow and its steps into your Next.js app's own routes, and in development WDK runs on a local backend with no extra infrastructure. The use-chat-wdk demo has a "Fail once" toggle and a "Crash" button: arm one, send a prompt, and the reply still settles exactly once while the process panel shows the retry.
What the client sees #
None of this changes the client. The same useChat
or useView
code from a single-process agent works unchanged against a durable one, because the durability lives in how the agent's output is published, not in what the client subscribes to. Over the Ably channel the client gets:
- A stream that persists on the channel, so refreshing the page mid-turn rehydrates the conversation and picks the stream back up.
- Catch-up from history for a client that joins late or reconnects after a drop.
- The same turn on every subscribed tab and device, in sync as it streams.
- Cancellation that travels over the channel: a client's stop publishes
ai-cancel
, the in-flight activity'sabortSignal
fires, and the run ends cleanly.
The framework keeps the execution alive across a crash. AI Transport keeps the conversation alive across a reconnect. The two compose because the framework's retryable unit and AI Transport's supersedable unit share one id.
Get started #
Install the SDK:
npm install @ably/ai-transport
Read the docsSee what AI Transport isRead the source, including the Temporal and WDK demosSign up free: you need an Ably account and an API key to run a session, and the free tier covers everything you need to start.