Every new chat window starts from zero.
I've been running coding agents (Claude Code, Cursor, Codex, and a rotating cast of others) long enough that this stopped being a minor annoyance and became the actual bottleneck. The agent isn't bad at the task. It's bad at remembering why the task looks the way it does — that we moved off approach A last Tuesday, that the flaky test is flaky for a known reason, that the config flag exists because of a specific incident.
So I spent a while trying to fix it. Then I spent longer fixing the fix.
1. Project rule files (CLAUDE.md, .cursorrules)
Works for static facts. Fails for anything that changes. The moment a decision gets revised, the rule file is stale and now actively misleading — and nothing tells you it's stale.
2. Pasting context manually
Reliable, until you forget. And you will forget, on the day you're moving fastest. Also: pasting 2000 tokens of background means those tokens are now competing with your actual question.
3. A very long system prompt
It grows. Then you start maintaining it like a codebase, except there's no test suite, so you can't tell what's still load-bearing. I removed a paragraph I was sure was useless and watched output quality drop.
The pattern in all three: I was asking the model — or the client — to be the memory. Neither of them is built for that.
Stop trying to make the model remember. Give it something to query.
Memory goes to disk, outside the context window. The agent gets tools to read and write it. Four responsibilities, kept separate:
Keeping those four apart was the single most useful architectural decision, because every failure below lands squarely in one of them.
1. The MCP server's lifetime is not yours
The MCP layer is a thin proxy; the actual memory service is a separate long-running process. If it isn't running, tool calls fail — and the error surfaces at the moment the agent needs memory most, which is exactly the wrong time to discover it. Treat the transport process as stateless and disposable, keep all state on disk, and make "service not running" a clear, actionable message rather than a stack trace.
2. "Remember everything" is worse than remembering nothing
My first version stored indiscriminately and dumped the top N results into context. Output got worse. Irrelevant memories don't just waste tokens — they actively mislead. Recall quality is a ranking problem, and ranking needs a relevance signal, not a recency sort.
3. Two agents writing at once
I run more than one agent. When two wrote to the same store concurrently, I got interleaved and occasionally contradictory records. Fixed with a single-writer discipline plus locking. Boring, unglamorous, non-optional.
4. Memories go stale and start contradicting each other
After a few weeks: "we use Postgres" and "we migrated off Postgres." Both true at their own timestamp, both useless to an agent that can't tell which is current. You need an explicit consolidation pass that detects conflict and supersedes — not just dedup by string similarity. This is the part I underestimated most.
5. Recall latency is on the critical path
Reading and ranking from disk on every turn adds visible delay. Cheap fix: maintain an index and impose a hard time budget on recall. If it can't rank in the budget, return fewer results rather than making the user wait.
6. Anything that reads project files will read .env
If a component ingests project files, it will eventually ingest a secret. Filter at the ingestion boundary, before it's written — not at read time, and not "later." Retrofitting a filter onto a store that already contains credentials is a genuinely bad afternoon. 7. Deciding what's worth ingesting is harder than reading it
The file-bridge is trivial to build and hard to tune. Reading a directory is easy. Knowing which files carry durable context — and which are generated noise — is the actual product.
8. Evaluation is the thing nobody budgets for
How do you know recall improved? I couldn't answer this for weeks, which meant "improvements" were vibes. What worked: keep a replay set of real past tasks and check whether the right memories surface for each. Without it, you're tuning blind.
These are decisions, not oversights, so I'll state them plainly:
| Decision | What it costs you |
|---|---|
| No cloud sync — memory is written to local disk only | Moving machines is manual |
| Retrieval runs locally, no cloud calls | No team-shared memory |
| Windows only right now — macOS is planned for a closed beta in Q1 2027 | Bad news today if you're on macOS |
The last one I'd change if I could. I can't yet.
The product is HyperMarrow. It's a local long-term memory layer for coding agents — the four modules above, with four ways to connect (MCP server, direct desktop integration, Python SDK, Node SDK). It runs on Windows.
There's a 30-day full-feature trial, with no email/password to set up — sign-in is by WeChat QR or SMS code. I'll be straight with you, because you'll find out anyway: that also means it currently expects a Chinese phone number, so if you're outside China the download is mostly useless to you. The eight failure modes above are the transferable part.
→ [https://hm.qianshi.cool/api/v2/dl?from=devto](https://hm.qianshi.cool/api/v2/dl?from=devto)
If a different approach works better for you, the eight failure modes above are the useful part of this post — they're the same whether you build it yourself or adopt something. Steal the checklist.