RocheDB: Data Locality as a First-Stage Retrieval Index

RocheDB, a new NoSQL database and vector-aware storage engine, introduces data locality as a first-stage retrieval index. By placing related data in nearby rings or locality groups, it reduces the candidate set before ranking algorithms like PageRank or vector search are applied. This approach aims to lower I/O amplification, memory pressure, and tail latency in search, recommendation, and RAG systems.

One of the ideas behind RocheDB is easy to miss if we describe it only as a NoSQL database, a document store, or a vector-aware storage engine for AI and search workloads. The stronger idea is this: Data locality and data placement can become the first stage of retrieval. In many databases and search systems, storage and indexing are treated as separate layers. Data is stored somewhere, then database indexes, inverted indexes, or vector indexes are built on top of it so that queries can find it later. RocheDB tries to push part of that work into placement. If related data is written into nearby rings, stellar neighborhoods, or application-defined locality groups, then the first retrieval question changes from: Which global index should I search? to: Which local region of the data should I open first? That difference matters for search infrastructure, recommendation systems, RAG systems, LLM context stores, vector search pipelines, and large retrieval-heavy applications. For teams operating large-scale retrieval systems, this is close to several daily engineering problems: candidate generation, candidate pruning, I/O amplification, memory pressure, cache locality, and tail latency before the ranking pipeline even starts. Search systems often discuss ranking algorithms and retrieval techniques: PageRank, BM25, vector similarity, freshness, authority signals, personalization, semantic search, reranking, and many other scoring methods. Those are important. But before ranking, a search or RAG pipeline has to decide what data even deserves to be examined. A simplified search, recommendation, or RAG retrieval pipeline looks like this: RocheDB focuses on the first two stages. It does not try to replace ranking algorithms. Instead, it tries to reduce how much unrelated data those algorithms, vector search systems, or LLM rerankers need to see. If the candidate set is already narrow and relevant, later ranking work becomes lighter. In a multi-stage ranking system, reducing weak candidates before the first ranking stage can also reduce downstream reranking cost and tail latency. Traditional database and search indexes are usually separate structures: Those structures are useful, and RocheDB is not arguing that they should disappear. But RocheDB adds another retrieval layer: The physical and logical location of the data is also retrieval information. A ring is not just a collection label. A ring says where the data belongs and where future search, RAG, or application reads should probably begin. For example: web/github/nim/rochedb/page web/github/nim/rochedb/keywords web/github/nim/rochedb/summary web/github/nim/rochedb/links web/github/nim/rochedb/embedding These records are not merely stored under similar names. They are intentionally near each other because a search or RAG request about RocheDB, Nim, or the project's documentation may need them together. That means the storage layout is already doing part of the candidate-selection, candidate-pruning, and working-set reduction work. Imagine a search engine, recommendation system, or AI search system that has already collected pages, links, summaries, keywords, embeddings, entity hints, language metadata, region metadata, and freshness information. One approach is to store those pieces separately and repeatedly join or look up the metadata needed for ranking, semantic search, or retrieval augmentation. Another approach is to keep related pieces close enough that the system can open a local region first, generate a smaller candidate set, and then rank what it finds there. RocheDB is designed for the second approach. The goal is not: Replace PageRank. The goal is: Give PageRank, BM25, vector search, semantic ranking, and rerankers less unrelated data to inspect. If unrelated candidates are not placed near the query's natural locality, they do not have to be opened first. That can reduce: RAG systems and LLM applications are especially sensitive to candidate volume. Even if the final answer only needs a few passages, the system may spend work on many documents before that point: If placement can reduce the candidate set before those steps, the savings are not limited to database read latency. They can propagate through the rest of the retrieval pipeline, reranking pipeline, and LLM serving path. This is why RocheDB's locality model is not only a storage concern. It is also an AI efficiency concern. In a RAG workload, reading less unrelated data can mean fewer tokens, less reranking work, smaller LLM context windows, less memory pressure, and lower LLM serving cost. At first glance, rings may look like database partitioning or sharding. There is overlap, but RocheDB is trying to use the idea more broadly. Partitioning or sharding usually answers: Which shard or partition owns this data? RocheDB rings and stellar neighborhoods also answer: Which nearby data is likely to be useful with this data? That second question is the important one. For search, recommendation, RAG, and vector retrieval, the expensive part is often not finding one exact row. It is candidate generation and candidate pruning: finding a useful local neighborhood without opening a large amount of weakly related data. This does not mean RocheDB should replace BigQuery, Snowflake, ClickHouse, or other analytical systems. Large cross-dataset aggregation is a different problem. If the task is to scan massive historical data and compute arbitrary analytics, a columnar warehouse or analytics engine is the right tool. RocheDB is aimed at operational databases, search backends, recommendation systems, RAG infrastructure, and retrieval-heavy paths where the application already knows meaningful locality: In those paths, many "aggregate", "related data", or "context retrieval" reads can be reduced to: Read this ring or nearby neighborhood, then summarize or rank it. That is a different shape from global ad-hoc analytics. The practical rule is simple: Do not put unrelated data near each other unless the application expects to retrieve it together. That sounds obvious, but many systems lose this information after ingestion. The application knows that data belongs to a user, tenant, topic, source, document group, vector-search namespace, or RAG corpus segment. Then later, retrieval has to rediscover that relationship through indexes, joins, filters, ranking features, or model-based reranking. RocheDB tries to keep that locality visible to the database. When writing data, the application or import rule chooses a ring: docs/japan/search users/123/context web/github/nim/rochedb products/42/reviews/2026 When reading data, that same ring becomes the first retrieval scope for search, recommendation, RAG, vector retrieval, or application reads. That is the core idea. The claim is not that every workload becomes faster automatically. The claim is narrower: If the storage layout captures useful data locality, then retrieval can avoid opening much of the unrelated dataset before ranking, vector search, reranking, or LLM context construction begins. That is why RocheDB benchmarks focus on working-set reduction, scanned records, candidate memory, RAG token reduction, and tokens/query, not only raw key-value latency. In other words, RocheDB is trying to make storage layout part of retrieval architecture. The target is not only faster reads, but less unnecessary work in candidate generation, pre-filtering, ranking, reranking, and context construction. Raw latency still matters. A locality-aware database cannot win if every local read is slow. But the larger bet is that reducing the amount of data touched can matter more than making a huge scan slightly faster. At small scale, reading extra data is often tolerable. At search-engine, vector-search, or AI-platform scale, "extra data" becomes expensive: If a system can avoid opening unrelated candidate regions in the first place, the benefit can be much larger than a small percentage optimization in one component. The useful mental model is: Data locality becomes the first-stage retrieval index. Ranking still matters. Search indexes still matter. Vector indexes still matter. Traditional database indexes still matter. But if the first stage can avoid reading most unrelated data, every later stage starts with a smaller problem. That is one of the main things RocheDB is trying to explore. Project repository: