{"slug": "giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one", "title": "Giving an AI Agent 8x faster Memory: What We Learned Building One", "summary": "A team building an on-call and triage agent for flight-booking failures reported that replacing Confluence as its memory store with a self-hosted Qdrant index plus an S3 cache delivered 8x faster search and 5x faster read, at the cost of a 5-minute staleness window on updates. The team chose self-hosted Qdrant over S3 Vectors specifically for hybrid search, because triage queries contain exact tokens such as error codes, booking IDs, supplier names and ticket keys that pure semantic search handles poorly. The write path still touches only Confluence, so the derived index can be rebuilt from the source of truth if the storage decision is revisited.", "body_md": "*Lessons from building the memory layer for an on-call and triage agent. Written for engineers who are about to build the same thing.*\n\nA large language model is stateless. Every run starts from a blank context window. For a chatbot that is tolerable. For an agent that does real work every day, it causes four problems.\n\n1. It repeats work it has already done. Take an on-call agent that triages a flight-booking failure. On Monday it spends 40 tool calls working out that a supplier timeout is the cause. On Tuesday it sees the same symptom and starts from zero. Nothing was learned.\n\n2. It repeats mistakes, and no one can correct it for good. A human reviewer says, “for payment failures, check the gateway status first, not the app logs.” Without memory that feedback lasts one conversation. Someone has to re-type it into a prompt forever.\n\n3. It cannot build on its own history. Questions like “has this happened before?”, “what did we conclude last time?” and “is this a new failure or a regression?” can only be answered if past runs are stored somewhere the agent can query.\n\n4. Context windows are not a substitute. Stuffing everything into the prompt gets expensive and slow, and quality drops as the window fills. The right facts have to be *retrieved* when they are needed. That is a memory system.\n\nThe pitch in one line: memory turns an agent from a function into a colleague who accumulates experience.\n\nThe vocabulary comes from cognitive science, and it maps well onto agents. Most confusion in design discussions comes from mixing these up, so it helps to name them.\n\nThree more forms sit alongside these. They aren’t separate cognitive categories, but you will meet them:\n\nIn our on-call triage agent we deliberately focused on two: episodic and procedural. *Episodic* is “what happened,” stored as reports. *Procedural* is “how to do it,” stored as playbooks and triage rules. Those two gave us most of the value. Semantic and user memory can come later.\n\nThe most useful question is: what is the agent going to do with this information?\n\nPractical rules of thumb:\n\nEarlier choice was to use confluence as a memory. But the read and search time was huge enough to motivate us and go for optimisation. The design goals were: durable and human-readable, fast enough for in-loop recall, and cheap to operate. With this we have achieved 8x faster search and 5x faster read.\n\nThe write path only touches Confluence. Everything else is derived, so any of it can be rebuilt from Confluence.\n\nS3 Vectors looked attractive: we already run on S3, there is nothing to host, and it is cheap. We still chose self-hosted Qdrant, for one reason: hybrid search.\n\nWhen S3 Vectors would be the right call: you only need semantic similarity, your queries are natural language rather than identifiers, you want zero infrastructure, or you’re at a very large scale where storage cost dominates. Choose based on whether your queries contain exact tokens. If they do, you need hybrid search.\n\nOne design consequence: because the index is derived data, this decision is cheap to revisit. If S3 Vectors gains sparse support, we can rebuild the index from Confluence without touching the source of truth.\n\nThe tension: we wanted memory that humans can read, edit and delete (Confluence), but an agent loop needs fast, predictable recall, and Confluence is neither fast nor built for machine-scale reads.\n\nThe S3 cache resolves this without giving up either goal:\n\nThe price is the 5-minute staleness window on updates (5.6). For agent memory, where notes are written once and recalled later, we judged that a good trade. fresh=True covers the cases where it isn't.\n\nThe pattern in general: keep a human-friendly system of record, and put a machine-friendly, disposable read layer in front of it. You get trust from the first and speed from the second.\n\nPure semantic search is bad at exact tokens: error codes, booking IDs, supplier names, ticket keys. Those are exactly what triage queries contain. Pairing dense embeddings with BM25 and fusing the results fixed most of the “why didn’t it find the obvious one?” cases. It also let us pick Qdrant over a vector-only store, because it supports dense, sparse and fusion natively.\n\nOur first instinct was to let the agent choose a path or parent page for each write. That failed in the obvious way: agents invent slightly different folder names, and the tree turns into a mess.\n\nThe fix was to make the server own placement. The agent passes domain, type and a title. The agent can't get the layout wrong because it never sees it.\n\nRule: give agents intent-level tools (“save this report for the domain”), not storage-level tools (“create a page under parent 4821”).\n\nAgents retry. Agents get triggered twice by the same alert. Without a guard you get five copies of the same report, and retrieval then returns five near-identical hits that crowd out everything else.\n\nWe added an optional dedup_key (for example id-<channel>-<ts>-<issue_type>), stored as a label. If a page with that label exists, the write updates it in place instead of creating a new one. It turns \"create\" into an idempotent upsert.\n\nTwo agents, or an agent and a human, editing the same page will silently overwrite each other. We pass expected_version on update, taken from a fresh read. If the page changed since, the update is rejected and the agent re-reads and merges. It's the same optimistic locking you'd use on any database row, and agents need it as much as people do.\n\nA cache makes stale reads possible, and an agent about to edit a page must not work from a stale copy. read(fresh=True) skips the cache and reads live from the source. Rule: reads for answering can be cached; reads that are the basis of a write must be fresh.\n\nWe chose eventual consistency on purpose, and the asymmetry is worth noticing:\n\nNobody will remember this in three months when someone asks “why didn’t my edit show up?” We wrote the exact sequence into the design doc, with the fix (a synchronous push on update) noted but not built. Documenting a known tradeoff costs an hour. Rediscovering it costs a day of debugging.\n\nThis one surprised us. Confluence’s CQL search is index-backed and lags new pages by minutes. Our “find or create the container page” logic used CQL, so it couldn’t see a container created a few seconds earlier and created a duplicate.\n\nThe fix was to use the non-index-backed endpoints (/content?spaceKey=&title=, /content?type=page) for anything that needs read-your-writes behaviour, and to keep CQL only where a lag is harmless. Any time you build dedup or find-or-create on top of a search API, check whether the search is eventually consistent.\n\n“What changed in the last 10 minutes?” never tells you what was *deleted*, because a deleted page simply stops appearing. If a human removes a bad memory, it must vanish from the cache and the index too, or the agent keeps recalling it forever.\n\nWe keep a manifest of {page_id: version, path} and diff it against the full list of live pages each tick. Anything in the manifest but not in the live set gets removed from S3 and Qdrant. It's a small feature, but without it the \"humans can curate memory\" promise is false.\n\nOur sync looks back 10 minutes but runs every 5. A window equal to the interval leaves no margin, so one late or failed tick loses changes. Overlap costs a little repeated work, which is fine because every step is idempotent: Qdrant point IDs are a deterministic hash of page_id:chunk_index, so re-syncing overwrites instead of duplicating. Design sync so that running it twice is always safe. Then retries, overlaps and crashes stop mattering.\n\nTitles change and pages get moved. We key S3 objects by page ID, and record the path in the search index payload and in an _ids/<page_id> pointer, so a rename never orphans anything. Reads stay O(1): a search hit carries its path, so a read is a single GET.\n\nThe agent-facing API is five operations: search, read, find, write, update. Tools return an error *string* rather than raising, so the agent can read the failure and adapt. find(domain, type, title) exists because \"give me the exact page called X\" is a different job from \"find things about X,\" and semantic search is the wrong tool for the first one.\n\nA memory system that silently stops syncing looks fine, because reads still work, until someone notices the agent is recalling week-old information. We post an alert whenever a sync tick has errors, and only then, so the alert stays meaningful. Silent staleness is the worst failure mode of a memory system.\n\n[Giving an AI Agent 8x faster Memory: What We Learned Building One](https://blog.devgenius.io/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one-777703c2c377) was originally published in [Dev Genius](https://blog.devgenius.io) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one", "canonical_source": "https://blog.devgenius.io/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one-777703c2c377?source=rss----4e2c1156667e---4", "published_at": "2026-11-16 12:55:56+00:00", "updated_at": "2026-11-16 13:24:03.692559+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-tools"], "entities": ["Confluence", "Qdrant", "S3 Vectors", "Amazon S3"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one", "markdown": "https://wpnews.pro/news/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one.md", "text": "https://wpnews.pro/news/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one.txt", "jsonld": "https://wpnews.pro/news/giving-an-ai-agent-8x-faster-memory-what-we-learned-building-one.jsonld"}}