{"slug": "i-ran-a-little-experiment-could-i-make-agent-memory-deterministic", "title": "I ran a little experiment: could I make agent memory deterministic", "summary": "LoreKit, a deterministic agent memory store, rejects vector embeddings in favor of a schema-based data model for coding agents, using scoped, keyed records with optional tags and TTL to ensure deterministic, inspectable, cheap, portable, and predictable memory. The design, detailed by its creator, prioritizes a WHERE clause over similarity search for small, recurring facts like 'tests need Postgres up first'.", "body_md": "Agent memory doesn't need embeddings — it needs a schema\n\nEveryone reached for vector databases. But most of what a coding agent needs to remember is small, specific, and recurring — a scoped note, not a semantic blob. Here's the deterministic, no-AI data model LoreKit runs on, the rationale for every column, and an invitation to converge on a shared shape.\n\nWhen agent memory got popular, almost everyone reached for the same thing: embed the text, store the vectors, query by similarity. Obvious move — it's how retrieval works everywhere else.\n\nFor a coding agent, it's mostly the wrong tool. The things an agent needs to remember aren't fuzzy documents you find by vibe. They're small, specific, and recurring: the integration tests need a database up first; this API returns [], not a 404; use the shared client, not a raw query. A scoped note, not a semantic blob. And for a scoped note, similarity search is a heavy, non-deterministic, opaque way to do what a WHERE clause does exactly.\n\nSo LoreKit went the boring way: a deterministic data model, no embeddings, no model call on the read path. This post is the rationale for every column in that model — and, at the end, an honest question: is a shape like this worth agreeing on?\n\nBefore the columns, the constraints they're chosen against. Agent memory for coding should be:\n\nDeterministic. The same store and the same task produce the same read, every time. You can reason about it, test it, and reproduce a bug.\n\nInspectable. It's plain records you can cat, grep, diff, and commit. When the agent recalls something wrong, you can see exactly which row and why.\n\nCheap. No embedding bill, no vector index to build or re-build, nothing to keep warm.\n\nPortable. It runs offline against files on disk, or against a hosted store, with the same shape. No service is required to get value.\n\nBoring. A junior engineer can predict what it will and won't surface. Predictability is a feature.\n\nEmbedding-based memory buys you real things — genuine semantic recall, paraphrase tolerance — at the cost of every item on that list. It's non-deterministic, opaque, metered, and drifts as you re-embed. For a pile of recurring, specific gotchas, that's a bad trade. So the design question is: what's the smallest set of columns that gets you most of the value, deterministically?\n\nA write is three required fields and a few optional ones:\n\n```\nmemory.write {\n  scope: \"repo::acme/checkout\",\n  key:   \"tests-need-local-postgres\",\n  value: \"Integration tests need Postgres up first: docker compose up -d db.\",\n  tags:  [\"ci\", \"gotcha\"],\n  ttl_days: 30                 // optional — auto-expire\n}\n```\n\nThat's the whole contract an agent has to honor. Everything else the store derives or keeps for you. Let me take the load-bearing fields one at a time — the question each answers, and what breaks without it.\n\nScope is the idea the whole model hangs on. A lesson is true somewhere specific: on this branch, in this repo, across a project, or everywhere. LoreKit encodes that as a canonical, collision-free address:\n\n```\nglobal\nproject::{name}\nrepo::{owner}/{repo}\nbranch::{owner}/{repo}::{branch}\n```\n\nThe :: double-colon is the only separator — deliberately, so a / in a repo path and a : in a branch name never get confused for structure. Scopes are read most-specific-first: a branch lesson beats a repo lesson beats a global one on the same key. That precedence is what lets a spike's findings stay on its branch while a hard-won global rule follows you everywhere.\n\nWithout scope you have one flat pile, and it gets worse as it grows — every project's quirks bleed into every read. Scope is how the same store serves a fresh repo and a shared, org-wide corpus without tuning.\n\nThe key is what makes memory a store and not a log. Write the same key twice and the second write updates the first — same lesson, one row. That's what keeps a store from filling with a thousand near-identical captures of \"the DB wasn't running.\" A good key is a slug for the lesson (tests-need-local-postgres), stable across the times you re-learn it.\n\nWithout a stable key you get append-only noise, and the signal you actually want — this keeps happening — is buried instead of counted.\n\nPlain text, and deliberately advisory. The read block that surfaces these literally labels them \"considerations, not rules.\" A lesson is a note your agent left for its future self; it can be ignored when it's wrong. That's the difference between lore and a linter — and why lore complements your CLAUDE.md rules rather than competing with them.\n\nScope says where; tags say what kind. They're orthogonal on purpose. A loop::reviewer-lessons tag buckets one automated writer's output; wip marks a throwaway; source::pr-webhook records that a lesson was harvested from a PR review. You can filter or cap by tag without touching the scope hierarchy. Collapsing the two axes into one — folders that mean both \"where\" and \"what\" — is exactly the tangle scopes-plus-tags avoids.\n\nBeyond what you write, the store maintains a few system columns. You don't set them; they're what make the read smart without a model in the loop.\n\nseen_count (recurrence). Every re-write of a key increments it. A lesson you've re-learned five times is worth more than a one-off, and this is the column that knows. It's the single most useful signal a deterministic store has — salience without semantics.\n\ncreated / updated (recency). Timestamps, decayed on a two-week half-life at read time. Recent lessons lean up; ancient ones fade without being deleted.\n\nexpires_at (TTL). Some facts are true for a week — \"skip the flaky test until Friday's fix.\" Set ttl_days on the write and the row goes invisible on its own. No cleanup task, no stale note steering you in November.\n\narchived_at (soft delete). Retiring a lesson hides it from reads without destroying the history.\n\nProvenance (source_agent, trigger, origin_repo / origin_branch / origin_commit / origin_pr). Where the lesson came from — which agent, what prompted it, the exact commit or PR. This is the audit trail: when a lesson is wrong, you can trace it to its origin instead of guessing.\n\nHere's the surprise: with those columns, you don't need embeddings to rank well. At the start of a task there's no query yet, so the read scores the candidates on signals the columns already carry — recurrence (seen_count), recency (updated), a cheap relevance pull from the branch name, and outcome (whether applying a lesson has helped, derived from its tags, with a neutral prior so a new lesson isn't buried). Equal weights by default, all deterministic, all inspectable.\n\nThat's retrieval from structure — scope precedence, a stable key, an honest recurrence count — rather than from vector similarity. Same store, same task, same result, every time, for a fraction of a cent.\n\nHere's the part I'm actually unsure about, and want dialogue on.\n\nEvery agent tool is quietly reinventing this shape — some file of remembered facts, addressed somehow, with some idea of freshness and recurrence. We're all solving the same small problem in mutually incompatible ways, so lore is trapped in whatever tool wrote it.\n\nI work with OpenTelemetry every day at Dash0, so I've watched this exact movie before. Telemetry used to be locked inside whatever agent emitted it, until a shared shape — spans, attributes, one wire format — let any tool read any other tool's data. Nobody had to win; they just had to agree on the shape. That's where the instinct here comes from: not one winning memory product, but one shape everyone can write to.\n\nThe part worth agreeing on, I think, isn't the storage engine or the ranking algorithm — those should vary freely. It's the two things that make lore portable: a record shape (scope, key, value, plus tags, ttl, provenance) and a scope grammar (global / project::… / repo::… / branch::…) that addresses a lesson the same way regardless of which agent wrote it. Get those shared, and a lesson your CI agent learns is one your editor agent can read — across tools, over MCP, without an import step.\n\nI'm not declaring a standard. I'm proposing a shape that has earned its keep in one real system, putting it out in the open, and asking the obvious questions:\n\nIs a deterministic, no-embeddings model enough for your use, or does it break somewhere I haven't hit?\n\nIs the four-level scope grammar the right set of levels — too many, too few, wrong names?\n\nWhat's missing from the record? What's there that shouldn't be?\n\nThe schema and the scope validator are open in the repo. Tell me where it's wrong — that's the point of writing it down.", "url": "https://wpnews.pro/news/i-ran-a-little-experiment-could-i-make-agent-memory-deterministic", "canonical_source": "https://www.lorekit.io/blog/agent-memory-data-model", "published_at": "2026-08-29 09:17:48+00:00", "updated_at": "2026-08-29 09:48:24.164557+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure"], "entities": ["LoreKit"], "alternates": {"html": "https://wpnews.pro/news/i-ran-a-little-experiment-could-i-make-agent-memory-deterministic", "markdown": "https://wpnews.pro/news/i-ran-a-little-experiment-could-i-make-agent-memory-deterministic.md", "text": "https://wpnews.pro/news/i-ran-a-little-experiment-could-i-make-agent-memory-deterministic.txt", "jsonld": "https://wpnews.pro/news/i-ran-a-little-experiment-could-i-make-agent-memory-deterministic.jsonld"}}