{"slug": "adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams", "title": "Adesh Nalpet Adimurthy open-sources PicoMQ for durable HTTP streams", "summary": "Adesh Nalpet Adimurthy, a Toronto-based founding engineer at Y Combinator-backed Terminal, has open-sourced PicoMQ, an Apache-2.0 server that provides durable, real-time HTTP streams with every record stored in S3-compatible object storage. The project, which appeared publicly on August 24 with 7 stars, 2 forks, 12 commits and four open issues, targets AI sessions and agent workflows by allowing developers to create separate streams for each session, device, or conversation, trading tens-of-milliseconds append latency for simpler operations.", "body_md": "# Adesh Nalpet Adimurthy open-sources PicoMQ for durable HTTP streams\n\n**PicoMQ trades tens-of-milliseconds durable appends for disposable nodes and independently addressable streams for sessions, devices and AI conversations.**\n\nBy [RuntimeWire Staff](/author/runtimewire-staff)\n· Published\n\nPrimary source: [PicoMQ](https://picomq.com/)\n\n## Why it matters\n\nAI sessions and agent workflows are creating demand for many small, resumable streams. PicoMQ tests whether open-source HTTP infrastructure can win developers willing to accept higher append latency for simpler operations.\n\nToronto engineer [Adesh Nalpet Adimurthy](https://github.com/addu390?ref=runtimewire) has published [PicoMQ](https://picomq.com/?ref=runtimewire), an open-source server that gives developers durable, real-time streams over HTTP while storing every record in S3-compatible object storage.\n\nPicoMQ appeared publicly on August 24, while its [repository records an initial commit](https://github.com/PicoMQ/picomq/commits/main/?ref=runtimewire) in the preceding few days. Adimurthy's GitHub profile lists Toronto and Terminal, where Y Combinator identifies him as a [founding engineer](https://www.ycombinator.com/companies/terminal/jobs?ref=runtimewire). Adimurthy also attended Dalhousie University, according to his [public profile](https://ca.linkedin.com/in/adesh-nalpet-adimurthy?ref=runtimewire).\n\nThe project is small by the usual measures of developer infrastructure. The [Apache-2.0 repository](https://github.com/picomq/picomq?ref=runtimewire) had 7 stars, 2 forks, 12 commits and four open issues on August 24. Its scope is considerably larger than those numbers suggest: Adimurthy is proposing a different unit for real-time systems, where developers create a separate stream for each session, device, workflow or AI conversation rather than combining unrelated records in a handful of large topics.\n\n### A smaller unit of streaming\n\nPicoMQ exposes streams as URL paths. A client sends `PUT`\n\nto create one, `POST`\n\nto append records and `GET`\n\nto read them. Long polling and server-sent events let readers follow new records, while offsets allow interrupted clients to resume. The current [HTTP API documentation](https://picomq.com/docs/api?ref=runtimewire) also describes idempotent producers, conditional appends, retention settings and sealed streams.\n\nThat interface is central to Adimurthy's bet. Kafka-compatible systems preserve an established protocol and its surrounding tools. PicoMQ instead treats an ordinary HTTP client as a complete streaming client. Its native Pico protocol supports record batches and numeric sequences, while the same engine can speak the open Durable Streams protocol through a separate listener, according to the [protocol documentation](https://picomq.com/docs/design/protocols?ref=runtimewire).\n\nThe intended granularity follows from PicoMQ's storage design. Each named stream is independently addressable and costs little while idle because the durable records live in object storage. PicoMQ's [introduction](https://picomq.com/docs/?ref=runtimewire) presents deployments ranging from ten streams to millions, with one stream assigned to an order, user session, device or job. That is an architectural target, rather than a production usage figure.\n\nFor AI applications, the appeal is straightforward. An agent conversation or workflow run naturally produces an ordered history that several processes may need to read, resume and audit. Giving each run its own URL avoids building a separate routing layer over shared topics. It also lets application developers use browser and server HTTP primitives instead of operating a conventional streaming client stack.\n\n### Every durable write goes to object storage\n\nPicoMQ separates record storage, coordination and compute. Object storage holds every record, including the write-ahead log. Postgres stores an ordered metadata command log in a cluster, while SQLite can handle metadata for a single-node installation. Server nodes hold caches and can be replaced without recovering unique local state, according to PicoMQ's [architecture overview](https://picomq.com/docs/design/overview?ref=runtimewire).\n\nThe SQL metadata log orders changes to the cluster's state. Each node tails that log and reconstructs the same in-memory view. Requests can land on any node; PicoMQ redirects a client when another node owns the requested stream. Epochs fence off stale nodes and producers.\n\nThis design removes the disk rebalancing associated with stateful brokers. PicoMQ says losing a node should produce seconds of rerouting rather than a data transfer between brokers. That remains a documented design claim. The young repository does not yet carry the production history that would establish how those transfers behave under sustained load, partial failures or large stream counts.\n\nPicoMQ is explicit about the cost of its choice. A durable append requires an object-storage round trip and typically takes tens of milliseconds. The software is therefore a poor fit for workloads demanding single-digit-millisecond acknowledgement. Adimurthy is offering operational simplicity and large stream counts in exchange for write latency, rather than hiding the trade behind a benchmark headline.\n\n### Authentication landed before a formal release\n\nPicoMQ ships as one Rust binary named `pico`\n\n, combining the server, client, administrative CLI, benchmark tool and embedded dashboard. The [repository README](https://github.com/PicoMQ/picomq?ref=runtimewire) documents a single-node setup using SQLite metadata and a local file-storage option, as well as clustered operation with Postgres and S3-compatible object storage.\n\nAs of August 24, PicoMQ had [no published GitHub releases](https://github.com/PicoMQ/picomq/releases?ref=runtimewire). Authentication appears in an [August 23 feature commit](https://github.com/PicoMQ/picomq/commit/4e03640443f5eb08e7619001703652c5b339c583?ref=runtimewire), so its behavior may differ across revisions.\n\nThe current documentation says authentication is off by default. PicoMQ [blocks non-loopback bindings](https://picomq.com/docs/operations/configuration?ref=runtimewire) unless an operator enables bearer-token authentication or explicitly permits an insecure remote bind. When authentication is enabled, PicoMQ stores SHA-256 hashes of opaque tokens in metadata and supports permissions scoped by stream prefix, operation and listener audience. Its [authorization documentation](https://picomq.com/docs/design/auth?ref=runtimewire) recommends terminating TLS at a proxy in front of the nodes.\n\nThose controls matter for software pitched at agent sessions, audit trails and per-user histories. A high-cardinality stream system also needs high-cardinality access control. PicoMQ's documented scopes can restrict a token to a stream prefix and place relative stream names under a tenant path.\n\n### Object storage is already crowded\n\nPicoMQ enters a category where several builders have reached the same underlying conclusion: broker disks are an expensive place to keep durable streaming data.\n\n[S2](https://s2.dev/blog/ga?ref=runtimewire) sells a managed service for durable streams over an API and object storage. In February 2026, S2 announced general availability and a $3.85 million seed round led by Accel, bringing its total funding to $5.5 million. S2 said customers were creating millions of streams and moving terabytes of data each week at the time of that announcement.\n\nElectric released [hosted Durable Streams](https://github.com/electric-sql/electric/blob/main/website/blog/posts/2026-01-22-announcing-hosted-durable-streams.md?ref=runtimewire) in January 2026 for persistent, resumable streams over HTTP, with AI sessions and multi-user applications among its stated uses. WarpStream takes a different path, retaining Apache Kafka compatibility while running a [diskless data plane](https://docs.warpstream.com/warpstream/overview/architecture?ref=runtimewire) against object storage. [AutoMQ](https://docs.automq.com/automq/architecture/s3stream-shared-streaming-storage/overview?ref=runtimewire) also preserves Kafka compatibility and adds a write-ahead-log layer intended to reduce the latency and API-call costs of object storage.\n\nPicoMQ's opening is self-hosted software with a plain HTTP resource model, disposable nodes and no required managed control plane. Its liabilities come from the same early-stage position: adopters must evaluate a new engine, a new native protocol and an operational record that is only beginning.\n\nAdimurthy has nevertheless chosen a clear boundary. PicoMQ does not promise Kafka compatibility or single-digit-millisecond writes. It offers a stream for every piece of application state that benefits from an ordered, resumable history. The next test is whether developers running agent workflows, device fleets and collaborative applications value that smaller primitive enough to put a new server in production.", "url": "https://wpnews.pro/news/adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams", "canonical_source": "https://runtimewire.com/article/adesh-nalpet-adimurthy-picomq-durable-http-streams-object-storage", "published_at": "2026-08-24 17:10:51+00:00", "updated_at": "2026-08-24 17:14:10.730091+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools", "artificial-intelligence"], "entities": ["Adesh Nalpet Adimurthy", "PicoMQ", "Terminal", "Y Combinator", "Dalhousie University", "Apache-2.0", "S3"], "alternates": {"html": "https://wpnews.pro/news/adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams", "markdown": "https://wpnews.pro/news/adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams.md", "text": "https://wpnews.pro/news/adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams.txt", "jsonld": "https://wpnews.pro/news/adesh-nalpet-adimurthy-open-sources-picomq-for-durable-http-streams.jsonld"}}