The diagnosis came down to our database architecture. We were reading from a read-replica to keep the main instance lean, but the write to the primary DB wasn't hitting the replica fast enough. The AI agent was querying the replica milliseconds after the user hit "Save," catching the stale version of the document.
I spent a few hours tracing the logs and found the discrepancy. The sequence looked like this:
-
User updates doc → Write to Primary DB (Success).
-
User asks question → AI Agent queries Read-Replica.
-
Read-Replica hasn't synced → AI receives old data → AI gives wrong answer.
To fix this, I had to implement a "read-your-writes" consistency check. Instead of blindly trusting the replica, the system now tracks the latest version timestamp of the document. If the replica is behind that timestamp, the agent is forced to route the query to the primary database.
if (replica.last_sync_time < document.updated_at) {
query_source = PRIMARY_DB;
} else {
query_source = READ_REPLICA;
}
This added a bit more load to the primary instance, but it completely eliminated the race condition. If you're building an AI workflow that relies on real-time data updates, don't trust "eventual consistency" when the LLM is the one doing the reading. It'll just lead to subtle, hard-to-debug errors.
Next AI Agent Runtime: Mapping Failure Points and Detection →