{"slug": "replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating", "title": "Replication Lag Is Why Your AI Agent Looks Like It's Hallucinating", "summary": "A developer argues that many apparent AI agent \"hallucinations\" in transactional workflows are actually replication lag: the agent's tool reads from an async read replica that hasn't yet applied a committed write, so the model reasons correctly over stale data. The writeup recommends techniques like WAIT FOR LSN and MySQL's WAIT_FOR_EXECUTED_GTID_SET, falling back to primary reads on timeout, and warns against faking freshness with LSN filters that return zero rows. It notes that read-your-writes fixes keyed on the writing user won't help when a separate dispatch agent performs the read.", "body_md": "The usual assumption: when an agent checks an order's status, sees `active`, and ships an order the customer cancelled a second earlier, the model hallucinated. Fix the prompt.\n\nIn a lot of transactional workflows that's the wrong layer. The cancellation committed on the primary. The agent's tool read from an async read replica that hadn't applied it yet. The model reasoned correctly over a row that was already out of date.\n\nSending agent, RAG and tool traffic to replicas is a sensible way to protect the primary. Fine for dashboards. For anything that reads a row and then acts on it, it's a trap: a human double-checks a stale dashboard, an agent just acts.\n\nWith async replication the primary confirms the write first and ships it to replicas afterwards. Until a replica applies it, reads there return the old value. AWS says Aurora replica lag is usually well under 100 ms, but it grows under heavy writes. Two things matter more than the typical size: it's never zero, and nothing guarantees an upper limit.\n\nKleppmann's *Designing Data-Intensive Applications* names the anomalies this causes. Three map straight onto agents: read-your-writes (your change goes missing), monotonic reads (two steps hit different replicas and time runs backward), and consistent prefix reads (a dispute shows up before its invoice).\n\nOne subtlety catches people. Read-your-writes is a promise to the client that wrote. When a customer cancels and a separate dispatch agent reads, a fix keyed on the user won't help.\n\n`WAIT FOR LSN` (brand new, still at release candidate); MySQL has `WAIT_FOR_EXECUTED_GTID_SET`. On timeout, your code reads the primary. Don't fake it with a `WHERE pg_last_wal_replay_lsn() >= ...` filter. That returns zero rows, and the agent concludes the order doesn't exist.`UPDATE ... WHERE status = 'active'`. Zero rows means don't dispatch.\nNone of this certifies the answer. A fresh read from the wrong table is still wrong, pinning adds load on the primary, and a stale cache in front of it all undoes half the work.\n\nThe full article has the diagrams, the Aurora caveats, the sources and the complete four-check audit for your agent tools: [https://abivarma.medium.com/the-replication-lag-trap-why-your-ai-agent-hallucinates-on-phantom-data-d19fee072790](https://abivarma.medium.com/the-replication-lag-trap-why-your-ai-agent-hallucinates-on-phantom-data-d19fee072790)", "url": "https://wpnews.pro/news/replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating", "canonical_source": "https://dev.to/abivarma/replication-lag-is-why-your-ai-agent-looks-like-its-hallucinating-k7f", "published_at": "2026-11-16 10:46:26+00:00", "updated_at": "2026-11-16 10:50:53.892985+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-safety"], "entities": ["Aurora", "MySQL", "PostgreSQL", "Martin Kleppmann", "AWS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating", "markdown": "https://wpnews.pro/news/replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating.md", "text": "https://wpnews.pro/news/replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating.txt", "jsonld": "https://wpnews.pro/news/replication-lag-is-why-your-ai-agent-looks-like-it-s-hallucinating.jsonld"}}