{"slug": "i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job", "title": "I tested Claude Code's memory against mine. They are not doing the same job.", "summary": "A developer who built a project-memory system for coding assistants tested it against Claude Code's built-in memory and found they serve different purposes. The built-in memory is bound to the vendor's harness, making one assistant continuous with itself, while the developer's system is bound to the repository, making knowledge about a codebase continuous across editors, machines, and people. The test revealed that semantic similarity and causal connection are different relations, and the developer's system adds a causal path to connect errors with fixes that vector search cannot.", "body_md": "*Proof over claim*\n\nLast time I described [the hour a platform shipped the feature I had spent months building](https://dev.to/heinrichneb/the-best-argument-against-my-mcp-server-came-from-anthropic-p1m). This is the part where I ran the test instead of the emotions, and found two things that were never competing.\n\nOne number up front, because it qualifies everything below: this is still one user. Every figure here comes from my corpus, my questions, my four servers. The method transfers. Whether the result does, I cannot tell you yet.\n\nThe test from last time is one question: teach it a fact only true in your world, close everything, come back in a fresh session, and count how often the fact comes back.\n\nBoth passed. That was the first surprise, and it is worth saying clearly: the built-in memory works. This is not a post about a competitor being bad.\n\nThe difference showed up when I changed one thing in the test. I asked the fact from a different editor.\n\nAnd then again from a different machine. And then I asked a colleague to ask it.\n\nThat is where the two answers stopped matching, and it had nothing to do with quality. It had to do with what each thing considers its own boundary.\n\nHere is the distinction I could not put into words on that bad evening, and it took a test to produce it.\n\nA vendor's memory is bound to the vendor's harness. It makes one assistant continuous with itself. That is genuinely valuable and it is what most people mean when they ask for memory.\n\nWhat I had built is bound to the repository instead. It makes the knowledge about a codebase continuous — across editors, across machines, across people, across model upgrades.\n\nThose are not two implementations of one feature. They are answers to two different questions. \"What did I just say?\" and \"what does this project know?\" only look similar until you switch tools.\n\nThe clearest way to see it: when someone leaves your team, a session memory leaves with them. A project memory does not, because it was never theirs.\n\nThe four boundaries above are about *where* memory lives. There is a fifth one, and it is about how you find anything in it — I only understood it because someone else wrote it down.\n\nThink about the two entries that matter most together: an error, and the fix that was found three weeks later. Write them out.\n\n`deploy hangs at \"Build image\", worker log says nothing`\n\n`the runner disk was full; docker prune and restart the service`\n\nThose two share almost no vocabulary. One is a symptom, one is a cause; one is about a build step, one is about a disk. In embedding space they sit far apart — not slightly, structurally. And they are the single most valuable pair in the whole store, because together they are the answer and apart they are two anecdotes.\n\nA similarity search cannot connect them. Not because the embeddings are bad. Because *semantic closeness and causal connection are different relations*, and only one of them is what a vector index measures.\n\nThat is why our store keeps a causal path beside the similarity one — which entry led to which, which contradicts which. I had treated that as a secondary feature for a long time. It is the part that similarity cannot do, and I needed an outside article to see it.\n\nThis also cuts against my own product, so I will say it plainly: if what you need is \"find me the thing that sounds like this\", a vector search over a file does that, and the extra machinery earns nothing.\n\nThis is the embarrassing part, and it is the reason I am writing it down. I had built features I did not use, because my own use case was one person and four servers.\n\nIt can hand knowledge between people. I had built the sharing path months earlier for a technical reason and never once used it, because I work alone on this. For a team it is the entire point: one person debugs a thing at 2 a.m., everyone else inherits the reason.\n\nIt can answer why, not just what. Every entry carries what worked and what failed. I had been reading only the first field for months. The second one is where the expensive knowledge lives — the approach that looked right and was not.\n\nIt can be asked from things that are not editors. Because it speaks MCP and HTTP, our own operations dashboard queries it: a question box that answers from the event stream plus the stored lessons. I built that as a side project and it turned out to be a second product surface.\n\nAnd it does not care which model you use. The lessons written by one model are read by the next one. I had treated that as an implementation detail. It is the reason the store survives an upgrade cycle that rewrites everything else.\n\nA comparison that only finds advantages is an advertisement, so here is the other direction, and I mean it.\n\nSetup: theirs is zero. It exists the moment you install. Mine needed a decision, an account and configuration — and measuring that gap honestly is what pushed us to a trial that needs no sign-up at all.\n\nDepth inside one conversation: theirs sits in the harness and sees everything. Anything from the outside sees what it is told, and that is a real ceiling, not a temporary one.\n\nTrust: the vendor already holds your code. Handing the same knowledge to a second party is an extra decision, and \"it is hosted in the EU\" is an answer, not a dismissal of the question.\n\nIf your work lives in one assistant, on one machine, alone, the built-in memory is the right choice and I would tell you so.\n\nThe test that produced all of this is four variations of one question, and the variations are the whole trick. Same fact, different boundary.\n\nRun it in the tool you taught the fact to. Then in a second editor. Then on a second machine. Then have someone else ask.\n\nEach of those four is a boundary a memory either crosses or does not, and no announcement will tell you which — the crossing is the thing that got designed, and it rarely shows up in release notes.\n\nWrite the four numbers down before you form an opinion. The shape of the four tells you which product you are holding.\n\nThe harness, extended from last time:\n\n``` bash\n#!/usr/bin/env bash\n# Four boundaries, one fact. The SHAPE of the four numbers is the answer.\nset -u\nFACT=\"which port the staging database listens on\"\nEXPECTED=\"5433\"           # what you stored, written down BEFORE you ask\n\nask() {  # ask <label> <command...>\n  local label=\"$1\"; shift\n  local hits=0\n  for i in 1 2 3 4 5; do\n    out=$(\"$@\" 2>/dev/null)\n    grep -qF -- \"$EXPECTED\" <<<\"$out\" && hits=$((hits + 1))\n  done\n  printf '%-22s %d/5\\n' \"$label\" \"$hits\"\n}\n\nask \"same tool\"      your-assistant --new-session --ask \"$FACT\"\nask \"other editor\"   other-editor-cli --ask \"$FACT\"\nask \"other machine\"  ssh other-box  your-assistant --ask \"$FACT\"\n# The fourth one is not scriptable, and that is the point:\n# ask a colleague to run the same question on their own machine.\n# ask \"other person\"  ...\n\n# 5/5 0/5 0/5 0/5  -> session memory. Continuous with itself.\n# 5/5 5/5 5/5 5/5  -> project memory. Continuous with the repository.\n# Neither is wrong. They are answers to different questions, and you now\n# know which one you have.\n```\n\nOne warning from our own numbers: do not measure recall quality by whether the answer sounds right. We once had a stored entry containing the exact address we needed, displayed at session start, and made the mistake anyway — the preview cut off at a hundred characters and the address sat at character three hundred and twenty-three. Delivery is a separate measurement from storage, and it is the one that decides whether any of this pays.\n\nBefore: a platform ships something adjacent to your work and you decide, from the announcement, whether you are finished. Half the time you are wrong in the pessimistic direction, which costs you the thing you were building.\n\nAfter: you run four variations of one test, get four numbers, and find out whether you were building the same product or a neighbouring one. It takes an afternoon and it replaces a week of dread.\n\nThe thing I actually learned is not about competition. It is that I had never described my own product, and a platform release forced me to — which turned out to be the most useful thing anyone did for it all year.\n\nI build **cachly** — memory for AI coding assistants, over MCP. ChatGPT and Claude remember your conversations; cachly remembers **your system**: the bug you fixed, why you chose Postgres, the deploy step that always breaks. Every assistant reads the same memory, and every lesson carries the name of whoever learned it.\n\n**Try it:**\n\n`npx @cachly-dev/mcp-server@latest demo`\n\nin any git repo. It reads your log locally and prints what an assistant would already know about the project. After npx fetches the package, the command makes no network calls.`/plugin marketplace add cachly-dev/cachly-mcp`\n\n, then `/plugin install cachly-brain@cachly`\n\n.`npx @cachly-dev/mcp-server@latest autopilot`\n\nwrites the MCP configuration for whichever assistant you use.", "url": "https://wpnews.pro/news/i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job", "canonical_source": "https://dev.to/heinrichneb/i-tested-claude-codes-memory-against-mine-they-are-not-doing-the-same-job-35jb", "published_at": "2026-09-01 20:40:47+00:00", "updated_at": "2026-09-01 20:54:55.529360+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "large-language-models"], "entities": ["Claude Code", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job", "markdown": "https://wpnews.pro/news/i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job.md", "text": "https://wpnews.pro/news/i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job.txt", "jsonld": "https://wpnews.pro/news/i-tested-claude-code-s-memory-against-mine-they-are-not-doing-the-same-job.jsonld"}}