RocheDB v0.5.0: Data Locality for RAG and LLM Retrieval

RocheDB v0.5.0, an open-source NoSQL document and vector store written in Nim, has been released with a focus on data locality. The new version introduces explicit locality at three levels: ring-based placement, stellar locality for reading related rings together, and WAL locality reporting for physical visibility. The project aims to make data locality a core part of the database model rather than an accidental result of indexes or caches.

RocheDB v0.5.0 has been released. Release: github.com/puffball1567/rochedb/releases/tag/v0.5.0 https://github.com/puffball1567/rochedb/releases/tag/v0.5.0 RocheDB is an open-source NoSQL document and vector store written in Nim. The project is built around one idea: Data locality should be part of the database model, not only an accidental result of indexes, caches, or application code. A lot of database performance discussions start with indexes, query syntax, or caches. Those are important. But layout often decides whether a system is working with the hardware or asking it to fight back. RocheDB v0.5.0 is a step toward making locality explicit at three levels: RocheDB uses a ring as a semantic and structural placement unit. An application, import rule, or operator chooses a ring when writing data: users/123/profile users/123/orders shops/1123/orders docs/japan/support tenant/acme/orders/2026 That ring is not only a directory-like label. It is a coordinate in the retrieval space. When a request already knows its natural locality, RocheDB can open that local region first instead of scanning unrelated records and filtering later. roche put --ring=docs/japan --payload='{"title":"Refund guide","status":"draft"}' --codec=json roche get --ring=docs/japan --filter='{"status":"draft"}' --selection='{ title }' The important part is not the string syntax. The important part is that placement and retrieval scope are connected. Point reads are important, but many real applications need related data. For example, a user page may need profile data, orders, billing information, and support metadata. In a relational database, that often becomes joins. In a document database, it often becomes manual denormalization or multiple application-side reads. RocheDB v0.5.0 adds a locality mechanism for this kind of shape: stellar locality . A stellar group is a read lens over nearby rings. It lets related rings be read together without pretending that all data must live in one document. roche stellar attach --stellar=user-123 --ring=users/123/profile roche stellar attach --stellar=user-123 --ring=users/123/orders roche stellar attach --stellar=user-123 --ring=shops/1123/orders roche get --stellar=user-123 The same stellar view can be narrowed: roche get --stellar=user-123 --subring=orders --limit=20 This is not meant to replace every join. It is meant to cover a different pattern: When the application already knows that several rings are useful neighbors, make that locality available to the database. A secondary index usually gives another path to find rows by a key. That is useful, but it can also fight the primary layout. The index finds the matching keys, then the database may still need to jump around to fetch the records themselves. RocheDB's current approach is more conservative. Secondary access should guide the query back toward ring-local reads where possible. In v0.5.0, stellar locality is not a global secondary index. It is a locality lens over rings that are already meaningful to the application. That keeps the model simple: roche get --stellar=user-123 --filter='{"status":"paid"}' --selection='{ title status }' This is why RocheDB is not just "routing by path." The ring path is part of the query plan, and stellar locality lets related ring coordinates become a retrieval unit. Logical locality is not enough by itself. If the storage layer gradually becomes fragmented, the first clean benchmark does not mean much. Real workloads mutate, delete, backfill, and query from odd angles. RocheDB v0.5.0 adds early physical locality visibility through WAL locality reporting: roche locality The goal is to make locality observable before building heavier compaction and layout optimization features. This is still early, but the direction is clear: There is also a locality layout demo: examples/locality layout demo.sh And a stellar data model demo: examples/stellar data model demo.sh RocheDB should not assume that writes are always clean. Applications backfill data. Imports arrive in imperfect order. Some records are attached to related data later. Some records are detached later. v0.5.0 adds attach/detach workflows for stellar locality: roche stellar attach --stellar=user-123 --ring=users/123/orders roche stellar detach --stellar=user-123 --ring=users/123/orders roche stellar list --stellar=user-123 This makes the locality model adjustable without rewriting the whole dataset. The current implementation is intentionally modest. It focuses on making relationships explicit and testable first. Heavier compaction and automatic relocation are better handled after the behavior is observable. RocheDB v0.5.0 also adds embedded atomic batch helpers: These are designed for application workflows where a group of related changes should either commit together or not commit at all. There are also cooperative coordinate locks: These locks are opt-in. Normal lightweight NoSQL reads and writes do not pay for them unless the application chooses to use them. The goal is not to turn RocheDB into a financial ledger. The goal is to support higher-integrity application workflows without losing the ring-first retrieval model. The main RocheDB benchmark claim is not "always faster than every database." The narrower claim is: If locality is meaningful, RocheDB can reduce how much unrelated data must be read, held, ranked, transferred, or passed downstream. The current benchmark documents include working-set, memory-pressure, RAG-oriented, PostgreSQL, and Redis comparison helpers. Some examples from the repository: These are local synthetic and generated benchmarks, not universal performance claims. The scripts are in the repository so the numbers can be reproduced, challenged, and improved. There are still important layout questions to solve. The biggest ones are: The current v0.5.0 work does not pretend those are solved forever. It gives RocheDB clearer mechanisms for observing and testing them. Many systems already know their useful locality: Often that information is known by the application but not directly usable by the database as a retrieval primitive. RocheDB tries to close that gap. It does not ask every workload to become a RocheDB workload. It is most interesting when the application can place data in meaningful local regions and then benefit from reading those regions directly. That is the direction of v0.5.0: make locality explicit, observable, and useful enough that it can become part of the database design itself. Repository: github.com/puffball1567/rochedb https://github.com/puffball1567/rochedb Release: github.com/puffball1567/rochedb/releases/tag/v0.5.0 https://github.com/puffball1567/rochedb/releases/tag/v0.5.0