There's a well-worn path to resumable AI chat streams: find the Vercel SDK docs, implement Redis-backed replay, and ship it. For many products, that's the right call.
The challenge arises when the product goes further than that. AI customer support tools that handle complex queries over 30-plus seconds. Agents that keep working while the user switches from their laptop to their phone. Products deployed to enterprise customers whose networks terminate long-lived Server-Sent Events (SSE) connections before the response arrives. For those applications, the page-reload case is solved. But everything around it isn't.
Almost every AI chat product, including the Vercel AI SDK's resumable stream feature, uses HTTP streaming (specifically SSE) as its transport layer: the protocol-level mechanism for delivering tokens from the agent to the user. HTTP streaming has no built-in concept of session recovery, so anything you need beyond raw delivery has to be built on top. Understanding what's happening at the transport layer is what makes sense of the Vercel approach, its gaps, and when you should consider an alternative.
Key takeaways #
- When an HTTP/SSE stream breaks mid-response, there is no built-in resume mechanism. The client must reconnect and restart inference unless the server has buffered the stream and assigned it a session identity.
- The Vercel AI SDK's chatbot resume streams feature handles page-reload resumption well. But it's single-device only, Next.js-coupled, and requires you to provision and operate Redis.
- A durable session decouples stream delivery from the connection itself. Any reconnecting client, on any device, picks up from the last delivered token without restarting the LLM call.
Why HTTP streaming breaks on disconnect #
HTTP streaming is stateless and point-to-point. When the connection closes, the server has no way of knowing whether the client will come back, has given up, or pressed stop. The client has no record of where the stream stopped: no offset, no sequence number, no reference it can use to resume.
Three common scenarios expose this limitation:
Page refresh. The client reconnects, but the server has either finished generating (and the response is gone) or is still streaming into the void with no delivery path. The user sees a blank response or watches the LLM restart from token zero.Network drop. The connection closes mid-token. The server may continue generating for some time before it detects the drop, burning inference cost with nothing to show for it. The client sees a frozen or truncated response.Tab switch or device change. The original connection is closed. A second device has no knowledge of the stream at all: no history, no offset, no way to join a session that was never persistent to begin with.
The first instinct is usually to re-issue the prompt automatically. But re-running inference produces a different response, costs another set of tokens, and tells the user something went wrong.
The Redis approach: what the Vercel AI SDK gives you #
To work around these HTTP/SSE limits, teams reach for a buffered replay model where tokens are persisted to a shared store as they're generated, so a reconnecting client can re-attach to the buffer instead of restarting inference. The Vercel AI SDK's chatbot resume streams feature is the implementation pattern most teams will encounter first, with Redis as the buffer.
The mechanism has five steps:
Message sent. The server generates a unique streamId and stores the live SSE stream in Redis via the resumable-stream package.ID persisted. The streamId is saved as the activeStreamId in a persistence layer.Client reconnects. The resume: true flag in useChat triggers a GET request to the server.Re-attach. The server looks up the active stream ID and re-attaches the client to the Redis stream from the current position.Cleanup. When generation finishes, the ID is cleared.
This gives you resumption on a full-page reload without re-running inference. And for many chat products, that's a meaningful improvement over nothing.
What it doesn't provide is everything around the mechanism. The Vercel SDK solves the resume hook and the stream attachment logic. The rest is infrastructure and integration work that has to be built and operated.
There are two categories to plan for. The first is operational responsibility: provisioning, the persistence layer, auth, and error handling. The table below shows what the SDK handles and what the developer owns for each.
| Capability | Handled by Vercel SDK | Developer responsibility |
|---|---|---|
| Stream publishing to Redis | Via consumeSseStream callback |
|
| Redis must be provisioned and operated | ||
| Resume on page reload | resume: true in useChat |
|
| GET endpoint must be implemented | ||
| Persistence layer | Stubs only (readChat /saveChat ) |
|
| Developer implements storage | ||
| Stop endpoint | Not provided | Developer implements; cancel logic required |
| Session auth | Not provided | Developer secures the resume GET endpoint |
| Error handling for Redis unavailability | Not provided | Developer handles expiry and failure |
The second is scope. Two framework-level constraints are worth knowing before you build:
Next.js coupling. The mechanism depends on Next.js's after() primitive (https://nextjs.org/docs/app/api-reference/functions/after) to keep tokens generating after the response closes. Express, Hono, and Fastify have no equivalent, and the docs don't cover what to do instead.Security is your responsibility. The GET endpoint that re-attaches a client to a stream has no auth implementation in the reference. Without adding it, any request with a valid stream ID can re-attach to the session.
For a chatbot answering short questions on a stable desktop, the Vercel SDK's resumable streams feature is the right level of investment. As production conditions get harder, that tradeoff starts to look different.
Where the DIY Redis path runs into trouble at scale #
So far, the Redis path has looked like solvable engineering: provisioning, persistence, auth, and the two framework-level constraints the SDK leaves to you. This section is about a different category of work. Four conditions that are technically solvable, but compound the infrastructure burden in ways that change the build-vs-buy calculation.
Write amplification. Resuming after a disconnect requires every token of every response to be persisted to a shared store as it's generated. You're paying to persist tokens for every conversation in order to support the small subset that drop. Batching the writes helps with throughput, but a client that reconnects mid-batch may ask for a token that hasn't reached the store yet.Zak Knill's analysis of SSE token streamingcovers the trade-offs in depth.Cancellation/disconnect ambiguity. The server can't tell a deliberate stop from a dropped connection. And the server handling the stop request often isn't the one running inference, so it can't cancel the inference directly. Instead, a cancel marker has to be written to a shared store, and the inference process has to check the store between tokens. Every response carries that check, even though only a small fraction are ever cancelled.Multi-device. The Redis stream is attached to a stream ID, not to a session that exists independently of the stream. A second device joining the same conversation has nothing to subscribe to without additional fan-out infrastructure.Enterprise network delivery. SSE relies on one long-lived connection from the server to the client, kept open for the full duration of the response. That's the kind of connection most enterprise networks will close before it finishes.AWS's Application Load Balancercloses idle connections by default.Cloudflare Workerscaps SSE at 30 seconds. Since SSE has no protocol fallback, and the Redis buffer only helps once the client reconnects, tokens that never reached the client can't be replayed.
Each of these is solvable with enough custom infrastructure. The question is whether you want to build and maintain it, or move to a transport layer that handles them as properties of how it works. A transport layer with durable sessions does that.
How a durable session handles AI chat stream resumption #
A durable session is the complete, persistent state of a conversation that exists independently of any participant. It outlives any single connection, so users can close their laptop, switch to a phone, or refresh the page without losing the stream.
This is structurally different from the Redis approach. The Redis buffer holds tokens at the transport layer, tied to whichever stream ID a client subscribed to. It's useful for replaying after a reconnect on the same device, but not reachable by other clients or devices. A durable session holds the conversation itself: every message, every turn lifecycle, any response in flight. Any authorised client can subscribe to that session at any time. There's nothing to fan out, no cancel marker to coordinate, no persistence schema to design.
Ably AI Transport is a transport layer designed for AI chat and agent workloads. It uses persistent WebSocket connections and pub/sub channels to make durable sessions a primitive: agents write to a session, clients subscribe to it, and the session itself outlives any connection. Reconnection, multi-device delivery, and resumption are properties of how the transport works, not features you build on top.
The integration into Vercel's useChat is small. You replace the default HTTP transport with the Ably transport, and the rest of your application code stays the same:
// Before: default HTTP transport
const { messages } = useChat();
// After: Ably transport (everything else stays the same)
// Wrap your tree with <ChatTransportProvider channelName={chatId}> first.
const { chatTransport } = useChatTransport();
const { messages } = useChat({ transport: chatTransport });
Ably handles delivery. Vercel handles UI rendering.
From the same integration, the session handles three states:
Connected. Tokens stream in realtime as they're generated.Reconnecting. The client picks up from the last token it received, in order and without duplicates. No retry logic to build.Returning from offline. The client loads the full conversation state on mount, including anything that happened while it was away.
Multi-device works the same way. A second client connecting to the same session sees the same state, whether the conversation is mid-response or complete. If a response is still streaming, the second client joins it in progress.
Session security is handled at the auth layer. Clients receive Ably tokens scoped to specific sessions via your auth endpoint. There's no separate resume endpoint to build or secure.
For longer-term offline cases (like notifying users that a response has completed while they're away from the app), Ably's Push API works alongside the session. See the push notifications guide.
When you need a transport layer with durable sessions, and when you don't #
The decision comes down to what UX you want to deliver, what your product needs to guarantee, and what you're willing to build and operate.
The Redis/Vercel SDK approach fits well for short responses, stable desktop connections, and single-device use, provided your team is comfortable operating Redis.
Durable sessions are a better fit when responses run 30 seconds or longer, users are on mobile or switch devices, or enterprise customers are likely behind corporate proxies. They also apply when multi-device/multi-tab continuity is a product-level requirement.
| Consider the Redis/SDK approach when... | Consider durable sessions when... |
|---|---|
| Responses that typically complete in under 30 seconds | Responses run 30+ seconds, and users need visibility of agent activity and health throughout |
| Users are on stable desktop connections with a single device per conversation | Users are on mobile, switch devices mid-conversation, or need multi-device/multi-tab continuity |
| Team is comfortable operating Redis | Team wants to avoid operating the session infrastructure |
| Page-reload recovery is sufficient | Tab switches, device changes, and enterprise proxy traversal matter |
| Single request, single response chats | Long-running chats where users need to interrupt or redirect the agent |
| No human-in-the-loop | Human-in-the-loop or human handover is a core requirement |
The honest tradeoff: Ably AI Transport is a new dependency. The integration itself is small, but it means working with durable sessions and Ably's auth flow in your application. Whether that's the right call depends on what your product actually needs the transport layer to do.
For a customer-facing AI support product, it's often the transport layer that the stack is missing. A session that breaks mid-conversation erodes trust. Users expect continuity across devices. Enterprise network environments are a real constraint.
The Vercel AI SDK's ChatTransport interface is the plug-in point. Ably drops in as the transport layer underneath.
Docs go deeper: Ably AI Transport reconnection and recovery, and the Vercel AI SDK framework guide.
Related reading: AI UX: reliable, resumable token streaming, and resume tokens and last-event IDs for LLM streaming.
Frequently asked questions #
What happens if a user disconnects during LLM streaming?
The server keeps generating, but the client has no delivery path. The tokens are either lost, or the conversation has to re-run from the start. A Redis buffer (the Vercel SDK approach) holds the tokens until the client reconnects, but only on the same device. A durable session holds them at the session itself, so any client that rejoins the session picks up where the conversation left off.
Can I resume an AI chat stream after a WebSocket reconnect without Redis?
Yes, but you need something that provides session identity and ordered token history. Redis is the most common DIY choice. Ably AI Transport provides both as properties of the session itself, without a Redis cluster to operate. The tradeoff is adding Ably as a dependency and integrating durable sessions into your application.
Does the Vercel AI SDK handle AI chat stream resumption on mobile?
The Vercel resumable-stream package covers page reloads on the same device. It doesn't cover mobile app backgrounding, OS-level connection kills, or a user switching from a mobile browser to a desktop. Those need a durable session that outlives the connection.
How do I prevent duplicate tokens when an AI stream resumes after a disconnect?
You need offset tracking. The client records the last token it received, and on reconnect requests delivery from that point. Without it, the client gets the full stream again from the start. Ably AI Transport handles this at the session level: the client tracks its position, and the platform delivers only what's missing.
What's the security risk with resumable AI chat stream sessions?
The main risk is session ID exposure. In a Redis-backed approach, anyone with a valid stream ID and access to the resume endpoint can re-attach to the session. The Vercel reference implementation leaves auth on the resume endpoint entirely to the developer. With Ably AI Transport, access requires an Ably token scoped to the specific session, issued via your auth endpoint. Auth is part of the session subscription, not a separate endpoint to build.