# Best memory for AI agents. Period. 98.7% recall, one Rust binary, no LLM

> Source: <https://dev.to/icegaming/best-memory-for-ai-agents-period-987-recall-one-rust-binary-no-llm-399h>
> Published: 2026-11-22 14:33:51+00:00

**Key takeaway:** Palace retrieves the right past session in the top 5 for 98.7% of LongMemEval questions using a 23 MB CPU embedding model at 6 ms per query, returning about 1,700 tokens instead of the full 115k-token haystack.

Every coding agent you use has the same flaw. It is brilliant for one session, and then it forgets everything.

You explain the architecture again. It suggests the fix you rejected last Tuesday. It greps 40 files to rediscover a convention you settled three months ago. It asks which test command you use. You told it. Yesterday. And the day before.

The industry answer has mostly been one of two things:

I think both are wrong. So I built [Palace](https://github.com/AncientiCe/palace-rs): local-first memory for coding agents. One Rust binary. One SQLite file. No API keys, no Python, no Docker, no vector database, no cloud.

And here is the part that surprised even me: it does not need an LLM to be good at this. On LongMemEval, the standard long-term memory benchmark, Palace finds the right session in the top 5 for 98.7% of questions, using nothing but a 23 MB embedding model on your CPU and 6 ms per query.

Let me show you how.

```
# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/AncientiCe/palace-rs/main/scripts/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/AncientiCe/palace-rs/main/scripts/install.ps1 | iex

palace install    # wires up every agent on this machine
palace doctor     # checks that each one is connected
```

Restart your agent. Then, the next day, in a brand-new session:

```
You:    why did we switch the queue to SQLite last month?
Agent:  [palace_search → wing my_api, room decisions, filed 2026-09-03]
        You dropped Redis because the deploy target had no persistent volume.
        The benchmark you ran is in the same drawer.
```

That's it. That is the product. Your agent remembers, and it tells you where it remembered it from.

It works with Claude Code, Codex, Cursor, Gemini CLI, VS Code, OpenCode, Cline, Zed, Goose, Windsurf and Claude Desktop. Eleven agents, one memory. A Codex session can pick up exactly where yesterday's Claude Code session left off, because they read from the same palace.

I'm going to lead with benchmarks, because "AI memory" is a category full of vibes and very short on reproducible results.

[LongMemEval](https://github.com/xiaowu0162/LongMemEval) gives a retriever a long chat history and asks questions about it. Every question comes with its own haystack of past sessions, and you have to find the session that holds the answer. Some questions need facts that were later updated. Some need reasoning about time ("what did I buy two weeks before the trip?"). Some need several sessions at once.

s_cleaned: ~50 sessions, ~115k tokens of haystack per question (470 questions, abstention excluded, the usual convention):

| Metric | Score | 
|---|---|
| Recall@1 | 0.91 | 
| Recall@5 | 0.987 | 
| Recall@10 | 0.996 | 

Per question type, at @5:

| Question type | R@5 | R@10 | 
|---|---|---|
| knowledge-update | 1.000 | 1.000 | 
| single-session-user | 1.000 | 1.000 | 
| single-session-assistant | 1.000 | 1.000 | 
| multi-session | 0.992 | 1.000 | 
| temporal-reasoning | 0.976 | 1.000 | 
| single-session-preference | 0.933 | 0.933 | 

Read that again: perfect recall@10 on knowledge updates, multi-session questions and temporal reasoning, the three categories that are supposed to be hard.

And it isn't just finding one of the right sessions. On questions that need several, recall_all@10 (every gold session in the top 10) is 0.938.

What comes back is tiny. The five drawers behind the @5 score hold about 1,700 tokens between them. That's 1.3% of the haystack. Your agent doesn't need a million-token context window to remember last month. It needs the right 1,700 tokens.

Then I made it ten times harder. m_cleaned has ~500 sessions and ~1.5M tokens of haystack per question, around 4,000 drawers to search for every single question:

| Metric | Score | 
|---|---|
| Recall@5 | 0.919 | 
| Recall@10 | 0.949 | 
| Avg query time | 78 ms | 

Over 9 in 10 questions answered from the top 5, out of a million and a half tokens, on a laptop CPU, in under a tenth of a second.

This is the part I want people to sit with:

A lot of memory systems put an LLM in the loop and call the result "intelligent memory". Every LLM call is latency, money, a privacy boundary you just crossed, and a lossy rewrite of what actually happened. Palace gets to 98.7% without one, and the retrieval is deterministic: run it twice, get the same answer.

The runner is public and the method is documented in [docs/benchmarks.md](https://github.com/AncientiCe/palace-rs/blob/main/docs/benchmarks.md). Please run it yourself. I would genuinely love to see the numbers reproduced on other hardware.

LongMemEval is about chat. Coding agents have their own kind of memory: decisions, past failures, commands, conventions, preferences. So Palace ships an eval built from exactly that, 51 questions like "why did we choose bundled sqlite?" and "what clippy command should I run?":

|  | R@1 | R@5 | 
|---|---|---|
| Palace (default model) | 0.922 | 1.000 | 

Every question answered from the top 5. It runs as a normal test: `cargo test --test coding_agent_eval -- --nocapture`.

On a real palace of 10,800 drawers:

| Indexing | 150 drawers/s |

| Query p95 | 2.0 ms |

I tried bigger embedding models (bge-small, bge-base). They were 2-4x slower to index, 2-3x slower to query, and no better on the coding eval. The smallest model won. Sometimes the boring answer is the right one.

There is no single trick. There are a lot of small, unglamorous decisions that compound.

Palace never rewrites your memory. A drawer is a chunk of the actual text, with its embedding and metadata. When an LLM summarises "we dropped Redis because the deploy target had no persistent volume" into "user prefers SQLite", the why is gone forever. Palace keeps the why.

Pure vector search is bad at exact identifiers, error codes, file paths and the name of that one flag. Pure keyword search is bad at paraphrase. Palace fuses BM25 with cosine similarity on every query, and it computes the real cosine for every keyword match instead of assuming zero for drawers outside the nearest-neighbour shortlist. (That one fix alone moved R@1 on a 12,500-drawer real palace from 0.345 to 0.430.)

Each query is classified: decision, how_to, preference, definition, temporal, or unknown. A preference question ("how do I like my commits?") gets a dedicated pass over tagged preference spans. A temporal question resolves "two weeks ago" against the date it's asked. A question that names code (an identifier, a path) ranks mined source and filed notes evenly; a plain-English question favours the decisions you and your agent actually filed.

Agents love to paste half a stack trace and three tool outputs into a search query. Palace sanitises the query before retrieval, so a 2,000-character dump doesn't drown the one sentence that matters.

Memory is organised like a building:

| Concept | What it is | 
|---|---|
| Wing | A project or topic (my_api, sales_notes) | 
| Room | An area inside it (backend, decisions, testing) | 
| Drawer | One verbatim chunk + embedding + metadata | 
| Knowledge graph | Time-stamped facts: my_api → uses_queue → SQLite (valid from 2026-09-03) | 
| Diary | Per-agent session notes for continuity | 
| Tunnel | A link between rooms in different wings | 

Facts in the knowledge graph have validity windows. When something changes, the agent invalidates the old fact and adds the new one, and the timeline keeps both. So "what did we use before SQLite?" is a question Palace can actually answer.

Here's the dirty secret of most MCP memory servers: they expose a search tool and then hope the model calls it. It usually doesn't. Models are trained to be helpful now, not to stop and check what happened last week.

Palace doesn't hope. `palace install` registers hooks:

| Hook | What it does | 
|---|---|
| Session start | Injects your standing rules, recent diary entries for this project (from any agent) and the project's most important drawers. | 
| Prompt submit | Searches the palace with your actual question and injects matching memory before the agent answers. | 
| Post tool use | While the agent greps and reads files, it surfaces related decisions, so the agent sees "we tried this in October and it broke prod" without ever searching. | 
| Stop | If the session did real work but saved nothing, it asks the agent once to write a diary entry and record decisions before it finishes. | 

That last one closes the loop. Memory only works if something gets written. Palace makes sure today's lessons become tomorrow's recall.

Every hook fails open. A missing or broken palace never blocks your agent.

Every search result carries its wing, room, source file, drawer id and a full score breakdown: combined, cosine, bm25, coding_boost, preference_match, intent, plus the chunks right before and after the hit in its source.

That means when your agent says "we decided X", you can check. No more confident hallucinations about decisions that never happened. If Palace said it, Palace can show you where.

And you can measure whether it's paying for itself:

```
palace gain
```

reports hit rate, precision@1 and @5, estimated tokens saved, repeat questions avoided, knowledge-graph facts recalled and p95 latency for your own sessions, from your own palace.db.

```
palace         ← the whole thing
palace.db      ← all of your memory
```

`cargo build --release`.` cp`.` io.github.ancientice/palace-rs`.
Want to mine an existing codebase?

```
palace init ~/my-project --yes --auto-mine
```

It detects rooms, writes a `palace.yaml`, and indexes the project. Re-running only touches what changed.

Coming from the Python MemPalace? `palace migrate` reads `~/.mempalace` (ChromaDB or the SQLite backend), keeps your timestamps, importance, diaries, knowledge graph and tunnels, and re-embeds everything. No Python needed.

Want to switch embedding models? `palace embeddings migrate <model>` rebuilds vectors in the background while search keeps working, then verifies the new model against 200 probes from your own palace and only switches if recall@5 doesn't drop and latency stays in budget. Rollback is instant. Your memory is never at the mercy of a model swap.

Your agent's memory is the most sensitive data you have about your work: architecture decisions, security fixes, incident notes, customer names in a debugging session, the half-formed idea you haven't told your team about yet.

With Palace, storage, embeddings and search all run on your machine. The only network calls are the one-time model download and, if you explicitly turn it on, your own self-hosted Palace Server for shared team memory. There is no telemetry to opt out of, because there is nothing to send.

No. CLAUDE.md is instructions you write by hand, and every line of it is loaded into every session. It doesn't grow on its own, it doesn't know which parts are relevant, and Codex can't read Claude's.

Palace is memory that grows as you work, recalls only what's relevant to the current question, and is shared across every agent you use. Keep your instruction file for rules. Use Palace for history.

The creative and personal profiles tune Palace for worldbuilding, fiction, coaching, caregiving and household notes. It reads Markdown and plain text, so Claude Desktop users can install the MCPB bundle and start without touching a terminal.

```
curl -fsSL https://raw.githubusercontent.com/AncientiCe/palace-rs/main/scripts/install.sh | sh
palace install
```

Then close your agent, open it again tomorrow, and ask it what you worked on today.

⭐ GitHub: [AncientiCe/palace-rs](https://github.com/AncientiCe/palace-rs)

📊 Benchmarks and method: [docs/benchmarks.md](https://github.com/AncientiCe/palace-rs/blob/main/docs/benchmarks.md)

💬 Using it? Tell me in [Show and tell](https://github.com/AncientiCe/palace-rs/discussions/categories/show-and-tell): which agents, and what it remembers for you. One line there helps more than a star.

Your agent is smart. It should also remember.
