Redpanda archived its GitHub repo

Redpanda introduced R2, a new C++23 coroutine-based engine that will power Redpanda Streaming, Redpanda SQL and upcoming products starting with version 26.3.1, and said Redpanda Streaming/Operator will no longer be source available from that release. Redpanda will keep a free Redpanda Community Edition usable up to 50 cores for companies of any size, and moved community discussions to github.com/redpanda-data/community. Redpanda said R2's formal TLA+ and Lean proofs, run on every core code check, caught a priority-inversion deadlock in GPU and accelerator memory quota management during an oversubscribed capacity run.

Redpanda named a Leader in G2’s Fall 2026 Event Stream Processing Reports Redpanda ranked 1 in implementation, usability, results, and relationship indexes The foundation every Redpanda data product will run on Redpanda has never shied away from reinventing the wheel when the road changes and we can do 10× better. The last time we did it, with the Apache Kafka® API, it ended up powering core banking in the US, AI labs, top retailers, chip manufacturing, and autonomous vehicles. For the engineer in me, this was a dream come true. Today we're introducing R2: a new engine of engines, and the foundation every Redpanda data product will run on from here forward. Starting with the upcoming 26.3.1, Redpanda Streaming https://www.redpanda.com/data-streaming , Redpanda SQL https://www.redpanda.com/sql and upcoming products will integrate R2 as the core engine. As a result, Redpanda Streaming/Operator will no longer be source available starting with 26.3.1. We will continue to maintain a Redpanda Community Edition free to use up to 50 cores for companies of any size. Community discussions moved to github.com/redpanda-data/community https://github.com/redpanda-data/community . R2 is the new engine that powers Redpanda's native products. Every design decision traces back to something production taught us over seven years of running Redpanda at scale for the most demanding workloads in the world, from space exploration and low-latency payments to embedded environments and high-volume write-through object storage. R2 is also a new programming model for us, designed for the age of agentic coding. Code generation is now cheap, but building an ultra-reliable product remains hard. Many of our users bet their business on Redpanda because we are often the ultimate system of record. A failure means data loss and a hit to their bottom line. The challenge is to harness advances in AI to accelerate innovation while strengthening the correctness, safety, and reliability that our customers' businesses depend on. R2 rests on four layers: std::expected<T, E so that all callers crash or handle 100% of the errors. All code is compiled with -fno-exceptions , so an unhandled exception cannot be lost. Most importantly, it lands us on one shared foundation after hundreds of years of collective engineering effort across Redpanda data products. It's the foundation we're building the company on. R2 is built from scratch using C++23 coroutines. It gives us one async abstraction over I/O, GPUs/accelerators, and files. Functionally, there's one local dev event loop across Mac and Linux, which is a much-improved developer experience. All products now share a single codebase. Each core runs 1 reactor on 1 OS thread, plus N alien threads N = 1 by default . Each reactor tick runs four phases: reap completions into work-class queues, run them with deficit-weighted round robin DWRR, up to 64 resumes per tick , flush the staged ops, then park until the next completion. The completion source is pluggable. On Linux it's io uring , on macOS it's kqueue . For testing, R2 uses a simulation source: a virtual clock, a seeded PRNG, virtualized I/O and seeded ordering, so that bugs can be replayed from seed. Some of the most important bugs were found in new code that hadn't been hardened in production yet, because the state space can be explored during code generation on CI instead of on live production systems. This is the first time we've made formal proofs part of every core code check, and the compounding effect is that all the code we've ported over has largely "just worked" and, in a few cases, even highlighted latent bugs in the original implementation. R2 uses a DWRR scheduler, so when we introduced quota management for GPUs and accelerator memory allocation, the TLA+ and Lean checks found a gnarly edge-case deadlock caused by priority inversion during an oversubscribed capacity run, even though the scheduler was correct. It took years of hardening to find these kinds of bugs inside R1, the current Redpanda Streaming engine, and this foundation has changed how we develop better software across all upcoming products. Coroutine suspension points are explicit, so under the simulation source every interleaving is enumerable and replayable from a seed. Paired with TLA+ specs and deterministic clocks, a timing bug that would take years of production traffic to hit shows up in minutes of simulated time. One of the most challenging parts of a thread-per-core architecture for new programmers is memory pressure. A seemingly large machine looks like a bunch of smaller computers with relatively limited memory. Our previous future scheduler split memory evenly across cores at boot and gave each shard/logical core its own allocator, with every cross-core free sent back to its owner. The fast path was cheap, but it made the small-computer problem literal. When a shard runs out of memory and cannot borrow from an idle neighbor, it forces the engineer to build sophisticated machinery for backpressure, memory reclamation, moving workloads dynamically across cores, etc. mimalloc , TCMalloc , and other general-purpose allocators have since caught up and, with much less co-design, perform well across a wide range of workloads, including slow-drip requests, ultra-low latency, and slow object storage writes with large arenas. R2 keeps thread-per-core for execution, but stops carving up memory by hand. We still do message passing from source to destination to enforce core-local execution and a single concurrency and parallelism model across the codebase. Every R2 binary links mimalloc : each thread allocates from its own heap with no lock, a block freed on another core goes onto its page's free list and returns when the owner next collects, and freed pages go back to the kernel after a short purge delay. We also ran TCMalloc in per-CPU mode. It's a good allocator, but mimalloc beat it on our workloads, so we removed it. Background alien threads for things like compaction or blocking syscalls are not pinned to a core, so the Linux CFS scheduler can move them. They leverage simpler reclamation and larger-than-logical-core allocations to simplify workload movement across CPUs and ship that logic to the Linux CFS scheduler, lowering latency for pathological workloads where most work moves to a single core and increasing system throughput. R1 https://www.redpanda.com/blog/redpanda-one-multi-modal-streaming-data-engine is, to the best of my knowledge, the first Kafka-API distributed log that delivered an ever-changing workload shape dynamically on a single cluster, with no detached consensus no ZooKeeper, no KRaft and every acknowledged write durable on disk by default. Safety, latency, throughput, consistency, availability, and networking costs are all tunable at the topic level. Nested control loops push the bottleneck to the next piece of hardware depending on the workload. It is the best engineering work any team I've been part of has ever shipped, and our customers' speed and scale test it every day. R1 stays on GitHub as is. R2 carries all of that forward starting with 26.3.1, and the new Redpanda Community Edition usage limits apply from that version on. Redpanda SQL https://www.redpanda.com/sql formerly Oxla is our Postgres-compatible analytical database. We acquired it to deliver a unified abstraction for streams and tables, grounded in the stream-table duality that Redpanda CTO, Tyler Akidau, helped formalize in One SQL to Rule Them All https://arxiv.org/abs/1905.12133 SIGMOD 2019 . No separate ETL pipelines, just SQL to consume, transform, and deliver data wherever it's needed. Since 2010, cores per socket grew 32× 8 to 256 while memory bandwidth per core grew only 2.5×, and fell to 3.2 GB/s on 2024's densest part. For analytical SQL, that bandwidth is now the cost-dominant factor, and we built Redpanda SQL around it. In a blocking design, a thread is two things at once: the unit of I/O concurrency and a single execution on a core. Those want opposite values. We rebuilt Redpanda SQL's S3 filesystem cache on R2's one-reactor-per-core runtime, then ported the whole query engine benchmarks below . We ported the whole query engine to R2 and ran it against Redpanda SQL today Oxla 1.4.0, default config on S3: same data, same EC2 m7a.8xlarge nodes, one engine at a time, on the Star Schema Benchmark and ClickBench, with every answer checksummed against the original. At 64 clients, the threaded version grew to 20,000–23,000 threads per node. The R2 port ran 35 threads per node at every level, 1 per physical core. Cold scans win the biggest because each core keeps many parked reads in flight, versus roughly 1 per core before. Not everything wins yet: warm ClickBench is level 1.02× excluding 1 regex GROUP BY query , and the port uses 20–30% more peak memory. Both are next on the profiler. R2 unifies all our native products: Redpanda Streaming, Redpanda SQL, and upcoming releases. It's also a new system to us, designed for the age of agentic coding, and it has already stopped non-trivial ~40 faults from making it to mainline. .alex Starting with 26.3.1, Redpanda Streaming/Operator is no longer source available, and Redpanda Community Edition stays free to use up to 50 cores for companies of any size. Community discussions will move to github.com/redpanda-data/community https://github.com/redpanda-data/community . Redpanda Shadowing now carries topic data, schemas, offsets, and ACLs on a single link for a low-risk, no-data-loss cutover Redpanda Operator 26.2 delivers simpler multi-region replication, safer rolling restarts, and deepens K8s support Subscribe to our VIP very important panda mailing list to pounce on the latest blogs, surprise announcements, and community events Opt out anytime.