Cloud & InfraArticle
The architectural trade-offs of stateful edge computing, and what it costs to build your own data plane.
Cloudflare's Durable Objects are often hailed as the ultimate primitive for stateful serverless. By combining V8 isolates with strongly consistent, colocated storage, they solve the hardest part of edge computing: state coordination. For collaborative document editors, multiplayer lobbies, and real-time chat apps, they are close to a perfect fit.
But as the industry shifts from lightweight web apps to heavy, data-dense AI agent workloads, the structural limits of this model are starting to show. When Wire, a platform that builds context containers for AI agents, announced they were moving their entire data plane off Durable Objects, it sent a clear signal. The very constraints that make Durable Objects secure, lightweight, and easy to manage are becoming bottlenecks for AI-native architectures.
For developers building the next generation of stateful edge applications, Wire's migration offers a masterclass in the trade-offs of the edge. It shows exactly where the isolate model breaks down, and what it costs to build your own alternative.
The Four Walls of the Isolate #
To understand why an AI-native application would outgrow Durable Objects, you have to look at how retrieval-augmented generation (RAG) actually works. Wire's context containers store processed knowledge, embeddings, and provenance graphs, which are queried over the Model Context Protocol (MCP).
When every container is a Durable Object, developers run into four structural limitations.
1. The Vector Indexing Wall
Retrieval is the most critical path for an AI agent. To do it quickly, the vector index must live next to the data. While Cloudflare recently brought native SQLite support to Durable Objects, the environment is highly sandboxed. You cannot load custom SQLite extensions.
Because of this, you cannot run in-process vector search libraries like sqlite-vec
. Developers are forced to store embeddings in a separate service, such as Cloudflare Vectorize. This introduces an extra network hop on the hottest query path and creates a second copy of the state that can easily drift.
2. The Compute Colocation Paradox
Retrieval is rarely a simple database lookup. It is a multi-stage pipeline involving hybrid candidate generation, fusion, query expansion, and wide reranking. In a Durable Object, the compute environment is too constrained to run this entire pipeline. Only a thin slice can execute where the data lives. The rest must be exported to external services, adding latency variance to the critical path of agent tool loops.
3. The Geographic Pinning Problem
Durable Objects are pinned to a geographic location near where they are first requested. As engineers at Clerk have noted, this design can lead to severe latency foot-guns. If an origin server in Ohio provisions a Durable Object, that object is pinned to a data center near Ohio. If a user in Seoul later tries to access that same object, their request must travel across the globe because only one active instance of that object can exist worldwide.
While location hints exist at creation, an object cannot dynamically migrate to follow its callers. Furthermore, because Durable Objects run on shared infrastructure, you cannot purchase dedicated capacity or guarantee process isolation for premium enterprise tenants.
4. The Self-Hosting Mandate
For enterprise software, compliance is often the ultimate gatekeeper. Regulated teams frequently demand the ability to run data containers on infrastructure they control. Because Durable Objects are tightly coupled to Cloudflare's proprietary global network, self-hosting is fundamentally impossible.
Rebuilding the Plane on Fly and Bun #
To break through these limitations, Wire rebuilt its container runtime from scratch. The new architecture shifts the data plane to Fly.io Fly Machines, running Bun as the host process.
Under this new model, each organization gets its own dedicated host process. A context container is no longer a Durable Object; it is a single SQLite file with the vector index embedded directly via sqlite-vec
.
+-------------------------------------------------------------+
| Fly.io Fly Machine |
| |
| +-------------------------------------------------------+ |
| | Bun Host Process | |
| | | |
| | +-------------------+ +-------------------+ | |
| | | Context Container | | Context Container | | |
| | | | | | | |
| | | SQLite File | | SQLite File | | |
| | | +-------------+ | | +-------------+ | | |
| | | | sqlite-vec | | | | sqlite-vec | | | |
| | | +-------------+ | | +-------------+ | | |
| | +-------------------+ +-------------------+ | |
| +-------------------------------------------------------+ |
+-------------------------------------------------------------+
By running the vector index in-process, candidate retrieval happens locally. The application can over-fetch and perform wide reranking before sending the final context to the LLM.
To solve the pinning problem, a custom per-region router places containers near the active caller. If a container goes idle, its state is snapshotted and shipped to object storage, allowing it to be rebuilt byte-for-byte on any machine in the world.
The performance gains of this architectural shift are stark:
xychart-beta
title "Container Wake and Tool Call Latency (Seconds)"
x-axis ["Idle Wake (Before)", "Idle Wake (After)", "Warm Tool Call (Before)", "Warm Tool Call (After)"]
y-axis "Seconds" 0 --> 4
bar [3.7, 1.4, 0.4, 0.3]
While a cold start for a raw V8 isolate takes milliseconds, reassembling a complex application stack on top of a Durable Object took Wire 3.7 seconds. Moving to dedicated Fly Machines cut that idle wake time to 1.4 seconds. More importantly, warm tool calls dropped from a highly variable 0.4 seconds (with spikes past 2 seconds) to a steady 0.3 seconds.
The True Cost of Leaving: Buying Back Durability #
If you are considering moving off Durable Objects, you must prepare to pay the engineering tax. Durable Objects are popular because they hand you durability and single-writer consistency for free. Cloudflare automatically replicates every storage write to multiple physical locations before acknowledging the write to the client.
When you build your own data plane, you inherit the responsibility of solving the CAP theorem.
In early test builds of the custom runtime, writes were only durable at the next scheduled checkpoint. When soak tests killed virtual machines under heavy load, acknowledged writes simply vanished.
To buy back the durability guarantees of Durable Objects, you must implement continuous Write-Ahead Log (WAL) shipping. In a custom SQLite setup, a write must not be acknowledged to the caller until its WAL frame is safely written to remote object storage. To prevent this from destroying write performance, you have to implement group commits, batching writes to keep the network overhead to roughly 100 milliseconds.
The Verdict #
This architectural shift does not mean Durable Objects are obsolete. If your application coordinates lightweight, short-lived state, like collaborative cursors, document editing, or API rate limiting, there is still nothing faster or more reliable to build on.
But if your application demands heavy, custom database engines, in-process vector search, or strict tenant isolation, the edge isolate model becomes a golden cage. The moment your compute needs to be heavier than a basic V8 sandbox allows, or your storage requires custom C extensions, it is time to start planning your exit strategy. Just make sure you are ready to write the code to keep your data safe when the machines go down.
Sources & further reading #
Why we're moving off Cloudflare Durable Objects— usewire.io - When the Internet Melted Down: Cloudflare’s Outage and the Magic of Durable Objects - DEV Community— dev.to - On Durable Objects | Kevin Wang’s Blog— thekevinwang.com - Overview · Cloudflare Durable Objects docs— developers.cloudflare.com - The Ultimate Guide to Cloudflare's Durable Objects— blog.ashleypeacock.co.uk
Ji-ho Choi· Security & Cloud Editor
Ji-ho covers the increasingly tangled overlap between cloud architecture and security, drawing on a background as a penetration tester to keep his reporting grounded in real-world attack paths. He never lets a vendor claim go unquestioned and insists that every buzzword come with a proof of concept.
Discussion 0 #
No comments yet
Be the first to weigh in.