PicoMQ trades tens-of-milliseconds durable appends for disposable nodes and independently addressable streams for sessions, devices and AI conversations.
By RuntimeWire Staff · Published
Primary source: PicoMQ
Why it matters #
AI sessions and agent workflows are creating demand for many small, resumable streams. PicoMQ tests whether open-source HTTP infrastructure can win developers willing to accept higher append latency for simpler operations.
Toronto engineer Adesh Nalpet Adimurthy has published PicoMQ, an open-source server that gives developers durable, real-time streams over HTTP while storing every record in S3-compatible object storage.
PicoMQ appeared publicly on August 24, while its repository records an initial commit in the preceding few days. Adimurthy's GitHub profile lists Toronto and Terminal, where Y Combinator identifies him as a founding engineer. Adimurthy also attended Dalhousie University, according to his public profile.
The project is small by the usual measures of developer infrastructure. The Apache-2.0 repository had 7 stars, 2 forks, 12 commits and four open issues on August 24. Its scope is considerably larger than those numbers suggest: Adimurthy is proposing a different unit for real-time systems, where developers create a separate stream for each session, device, workflow or AI conversation rather than combining unrelated records in a handful of large topics.
A smaller unit of streaming
PicoMQ exposes streams as URL paths. A client sends PUT
to create one, POST
to append records and GET
to read them. Long polling and server-sent events let readers follow new records, while offsets allow interrupted clients to resume. The current HTTP API documentation also describes idempotent producers, conditional appends, retention settings and sealed streams.
That interface is central to Adimurthy's bet. Kafka-compatible systems preserve an established protocol and its surrounding tools. PicoMQ instead treats an ordinary HTTP client as a complete streaming client. Its native Pico protocol supports record batches and numeric sequences, while the same engine can speak the open Durable Streams protocol through a separate listener, according to the protocol documentation.
The intended granularity follows from PicoMQ's storage design. Each named stream is independently addressable and costs little while idle because the durable records live in object storage. PicoMQ's introduction presents deployments ranging from ten streams to millions, with one stream assigned to an order, user session, device or job. That is an architectural target, rather than a production usage figure.
For AI applications, the appeal is straightforward. An agent conversation or workflow run naturally produces an ordered history that several processes may need to read, resume and audit. Giving each run its own URL avoids building a separate routing layer over shared topics. It also lets application developers use browser and server HTTP primitives instead of operating a conventional streaming client stack.
Every durable write goes to object storage
PicoMQ separates record storage, coordination and compute. Object storage holds every record, including the write-ahead log. Postgres stores an ordered metadata command log in a cluster, while SQLite can handle metadata for a single-node installation. Server nodes hold caches and can be replaced without recovering unique local state, according to PicoMQ's architecture overview.
The SQL metadata log orders changes to the cluster's state. Each node tails that log and reconstructs the same in-memory view. Requests can land on any node; PicoMQ redirects a client when another node owns the requested stream. Epochs fence off stale nodes and producers.
This design removes the disk rebalancing associated with stateful brokers. PicoMQ says losing a node should produce seconds of rerouting rather than a data transfer between brokers. That remains a documented design claim. The young repository does not yet carry the production history that would establish how those transfers behave under sustained load, partial failures or large stream counts.
PicoMQ is explicit about the cost of its choice. A durable append requires an object-storage round trip and typically takes tens of milliseconds. The software is therefore a poor fit for workloads demanding single-digit-millisecond acknowledgement. Adimurthy is offering operational simplicity and large stream counts in exchange for write latency, rather than hiding the trade behind a benchmark headline.
Authentication landed before a formal release
PicoMQ ships as one Rust binary named pico
, combining the server, client, administrative CLI, benchmark tool and embedded dashboard. The repository README documents a single-node setup using SQLite metadata and a local file-storage option, as well as clustered operation with Postgres and S3-compatible object storage.
As of August 24, PicoMQ had no published GitHub releases. Authentication appears in an August 23 feature commit, so its behavior may differ across revisions.
The current documentation says authentication is off by default. PicoMQ blocks non-loopback bindings unless an operator enables bearer-token authentication or explicitly permits an insecure remote bind. When authentication is enabled, PicoMQ stores SHA-256 hashes of opaque tokens in metadata and supports permissions scoped by stream prefix, operation and listener audience. Its authorization documentation recommends terminating TLS at a proxy in front of the nodes.
Those controls matter for software pitched at agent sessions, audit trails and per-user histories. A high-cardinality stream system also needs high-cardinality access control. PicoMQ's documented scopes can restrict a token to a stream prefix and place relative stream names under a tenant path.
Object storage is already crowded
PicoMQ enters a category where several builders have reached the same underlying conclusion: broker disks are an expensive place to keep durable streaming data.
S2 sells a managed service for durable streams over an API and object storage. In February 2026, S2 announced general availability and a $3.85 million seed round led by Accel, bringing its total funding to $5.5 million. S2 said customers were creating millions of streams and moving terabytes of data each week at the time of that announcement.
Electric released hosted Durable Streams in January 2026 for persistent, resumable streams over HTTP, with AI sessions and multi-user applications among its stated uses. WarpStream takes a different path, retaining Apache Kafka compatibility while running a diskless data plane against object storage. AutoMQ also preserves Kafka compatibility and adds a write-ahead-log layer intended to reduce the latency and API-call costs of object storage.
PicoMQ's opening is self-hosted software with a plain HTTP resource model, disposable nodes and no required managed control plane. Its liabilities come from the same early-stage position: adopters must evaluate a new engine, a new native protocol and an operational record that is only beginning.
Adimurthy has nevertheless chosen a clear boundary. PicoMQ does not promise Kafka compatibility or single-digit-millisecond writes. It offers a stream for every piece of application state that benefits from an ordered, resumable history. The next test is whether developers running agent workflows, device fleets and collaborative applications value that smaller primitive enough to put a new server in production.