{"slug": "motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots", "title": "moteDB 0.5.1 Is Out: What 18 Months of Building an Embedded Database for Robots Taught Me", "summary": "MoteDB 0.5.1, an embedded database for robots, has been released after 18 months of development. The update introduces atomic epoch-bounded reads to prevent data gaps during buffer drains, incremental DiskANN for online vector index builds, and transaction support for atomic multi-index writes. These changes address real-world issues such as robots failing to recognize objects due to read-drain race conditions.", "body_md": "Last Tuesday our warehouse robot forgot what a red box looked like.\n\nNot metaphorically. It had the force curve, the timestamp, the GPS pose — everything except the one thing that mattered: the visual embedding that let it recognize the same box three minutes later. The data was there. It just couldn't be *queried* the way the task needed.\n\nThat's the bug class moteDB exists to kill. And 0.5.1, released this week, is the version where I finally feel like we got the architecture right.\n\nIf you've been following since the v0.1 / v0.2 days, this release is the one where a lot of the awkward scaffolding comes down. Here's what actually changed — and what I'd tell past-me if I could.\n\n`motedb 0.5.1`\n\nis on [crates.io](https://crates.io/crates/motedb) right now. `cargo add motedb`\n\nor pin `motedb = \"0.5.1\"`\n\n.\n\nIt's the same 100% Rust, serverless, multimodal embedded database it always was — vectors, time-series, scalar, and text in one file, one process, no daemon. What's different is *how* the write path and the index path behave under real robot loads.\n\nThe headline changes since v0.2:\n\nLet me unpack the ones that actually changed how I think.\n\nIn v0.2, the two-layer design (in-memory buffer → on-disk B-tree with a `drain_lock`\n\n) fixed the 3-hour benchmark hang. But it had a quiet flaw: when the active buffer flipped to immutable and the drain thread picked it up, **new writes could still observe a half-drained state** if they raced the drain. For sensor data at 200Hz, that meant occasional \"the reading exists but the column index can't see it yet\" gaps.\n\n0.5.1 makes the buffer flip atomic at the WAL level. The trick is boring but effective: every write is stamped with an epoch counter, and the drain thread publishes its completed epoch *before* marking the buffer immutable. Reads filter by `(buffer_epoch <= read_epoch)`\n\n, so a query never sees a buffer that's mid-migration.\n\n``` js\n// v0.5.1 — reads are epoch-bounded, never mid-drain\nlet visible = db.query()\n    .scalar_range(\"force\", 2.0.., 8.0)\n    .with_epoch(read_epoch)   // new in 0.5\n    .execute()?;\n```\n\nThe robot that forgot the red box? That was a read crossing a drain boundary. It doesn't happen anymore.\n\nThis was the thing I was most annoyed about for a year. DiskANN is the right call for edge hardware — memory-mapped, bounded query memory, no 1.5GB HNSW graph in RAM. But v0.2 only supported *offline* index builds. Add 50k new vectors from a day of operation and you rebuilt the whole graph. On a Pi, that's hours.\n\n0.5.1 ships **incremental DiskANN**. New vectors get inserted into the existing on-disk graph with local neighbor pruning; the global graph quality slowly converges as the robot operates. You pay a small recall tax during the warm-up window, then it settles.\n\nI was skeptical this would hold up under deletes (graph splits were the panic source in early testing). 0.5.1's delete path marks nodes tombstoned and re-prunes neighbors lazily during background compaction instead of mutating the live graph. That fixed the crash and kept query latency flat.\n\nThe whole point of moteDB is \"find the camera frames where force exceeded 2N in the last 3 seconds, ranked by similarity to this query image.\" That's three indexes — vector, scalar, time-series — touched in one query.\n\nIn v0.2 they shared a WAL, so a crash rolled them back consistently. But a *normal* multi-write sequence — insert vector, insert scalar, insert text — had no transaction boundary. If your process died between step 2 and 3, you had an orphaned vector with no description.\n\n0.5.1 adds `db.transaction()`\n\n:\n\n``` js\nlet tx = db.begin()?;\ntx.insert(b\"frame_0042\", &embedding)?;\ntx.insert_scalar(b\"frame_0042\", \"force_curve\", &curve)?;\ntx.insert_text(b\"frame_0042\", \"desc\", \"red box, slightly torn corner\")?;\ntx.commit()?;   // all three visible, or none\n```\n\nFor a robot that logs 200 things per second, atomicity isn't a nice-to-have. It's the difference between \"the memory is trustworthy\" and \"the memory is a liability.\"\n\nThis is the one I'm most excited about, and the most overdue.\n\nA robot's natural unit of memory isn't \"a row\" — it's **an episode**: everything that happened between \"start picking up the red box\" and \"placed it on the shelf.\" Sensors, frames, force curves, the plan, the outcome. One bounded, replayable unit.\n\n0.5.1 adds `Episode`\n\n:\n\n``` js\nlet ep = db.episode(\"pick_red_box_001\")?;\nep.log_frame(&embedding, now)?;\nep.log_scalar(\"wrist_torque\", torque, now)?;\n// ... the whole task ...\nep.close()?;  // seals the episode, builds its local index\n```\n\nLater you replay it, diff two episodes, or query \"show me every episode where wrist torque spiked.\" It's just structured sugar over the indexes we already had — but sugar that matches how embodied AI actually thinks.\n\nThree things, in order:\n\nNot everything is a win, and I won't pretend it is:\n\n```\ncargo add motedb@0.5.1\n```\n\nDocs and examples are on [GitHub](https://github.com/motedb/motedb). The test suite now runs 1,200+ cases under concurrency in about 4 minutes with zero hangs.\n\nIf you're building anything that needs to *remember* in the physical world — robots, drones, edge agents — I'd genuinely like to hear what your memory layer looks like today. What's the one query you wish your stack could answer that it can't?", "url": "https://wpnews.pro/news/motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots", "canonical_source": "https://dev.to/motedb/motedb-051-is-out-what-18-months-of-building-an-embedded-database-for-robots-taught-me-hkj", "published_at": "2026-07-11 05:06:06+00:00", "updated_at": "2026-07-11 05:10:58.392587+00:00", "lang": "en", "topics": ["developer-tools", "robotics", "ai-infrastructure"], "entities": ["moteDB", "DiskANN", "Rust", "crates.io"], "alternates": {"html": "https://wpnews.pro/news/motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots", "markdown": "https://wpnews.pro/news/motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots.md", "text": "https://wpnews.pro/news/motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots.txt", "jsonld": "https://wpnews.pro/news/motedb-0-5-1-is-out-what-18-months-of-building-an-embedded-database-for-robots.jsonld"}}