Everyone wants to talk about the AI part. Nobody wants to talk about the part where an invoice update arrives twice because a webhook retried, and now the agent has “remembered” the same payment outcome three times. That's the part I owned, and it turns out it matters more than the prompt.
PayEcho recommends how to recover an overdue invoice — which channel to use, when to follow up, whether a customer's history should factor into a new credit decision — by recalling what actually happened with that customer before. That recall only works if the data feeding it is correct, deduplicated, and current. My piece was the backend: the schema, the sync pipeline that turns transactional events into memories, and the API layer the dashboard and the recommendation agent both depend on. It's easy to think of Hindsight as “where the AI part lives” and the backend as plumbing. In practice, the plumbing decides what the AI part is even allowed to know.
01
Two stores, one truth
PayEcho keeps two separate stores, on purpose. A relational database holds the transactional truth — customers, invoices, payment records, recovery actions, timestamps. Hindsight holds the interpreted version of that truth: short narrative memories tied to a customer, built for recall, not for joins.
CREATE TABLE recovery_attempts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
customer_id UUID NOT NULL REFERENCES customers(id),
invoice_id UUID NOT NULL REFERENCES invoices(id),
channel TEXT NOT NULL,
sent_at TIMESTAMPTZ NOT NULL,
responded_at TIMESTAMPTZ,
outcome TEXT,
synced_to_memory BOOLEAN NOT NULL DEFAULT FALSE
);
That synced_to_memory flag is the one column that made the rest of the pipeline sane. Without it, I had no reliable way to answer “has this event already become a memory,” and the agent started seeing duplicate versions of the same outcome — which, if you're not paying attention, looks exactly like the agent being more confident about a pattern than it actually is.
02
Deciding when a row becomes a memory
The interesting backend problem was never storing the data; it was deciding when a transactional row becomes worth writing to memory, and how to phrase it. A row only becomes a memory once its outcome is final, since a reminder that hasn't been responded to yet isn't a story worth telling the agent:
def sync_pending_attempts():
attempts = db.query(
"SELECT * FROM recovery_attempts "
"WHERE outcome IS NOT NULL AND synced_to_memory = FALSE"
)
for attempt in attempts:
days = (attempt.responded_at - attempt.sent_at).days if attempt.responded_at else None
memory.retain(
agent_id="payecho-recovery",
user_id=str(attempt.customer_id),
content=build_narrative(attempt, days),
metadata={
"invoice_id": str(attempt.invoice_id),
"channel": attempt.channel,
"outcome": attempt.outcome,
},
)
db.execute(
"UPDATE recovery_attempts SET synced_to_memory = TRUE WHERE id = %s",
(attempt.id,),
)
build_narrative() is a small function, but it's the one that decides what the agent actually gets to reason over — it turns a row into a sentence like “sent a reminder via WhatsApp, customer responded after 3 days, outcome: paid.” The sync job runs on a schedule, picks up anything finalized and unsynced, writes it, and only then flips the flag. If the job crashes mid-batch, nothing gets double-written, because the flag update happens per row, right after that row's memory write succeeds, rather than as one commit at the end of the batch.
■ TERMINAL — the workspace the sync job and memory panel run in day to day.
03
One API, not two
The dashboard and the recommendation agent both need customer data, but they need different shapes of it — the dashboard wants aggregates like outstanding, overdue, and recovered totals, while the agent wants a specific customer's recalled memories plus their live invoice status. Rather than let two parts of the system fetch that separately, I built one internal API that both read from, so they never drift out of sync with each other:
@app.get("/customers/{customer_id}/context")
def get_customer_context(customer_id: str):
invoices = db.get_open_invoices(customer_id)
memories = memory.recall(
agent_id="payecho-recovery",
user_id=customer_id,
query="past recovery attempts, channel responses, and payment outcomes",
)
return {
"open_invoices": invoices,
"recalled_history": memories,
}
This one endpoint feeds both the customer detail page and the recommendation prompt. Early on I had two separate code paths doing roughly the same fetch, and they'd occasionally disagree — the dashboard would show an invoice as paid while the agent's recall still referenced it as outstanding, because one path was reading a cache the other wasn't. Collapsing them into a single endpoint got rid of that class of bug entirely.
■ EXPECTED OUTPUT — the same customer context, read from the one endpoint, behind both the dashboard and the recommendation.
04
The case that proved it
The clearest proof this backend work paid off showed up in a case that had nothing to do with prompt wording at all. A payment came in through a bank reconciliation import, not through the WhatsApp or email flow the agent was watching. Because the reconciliation import wrote to the same invoices table and went through the same sync job, the agent picked it up on the next recall exactly like any other outcome, and stopped recommending follow-ups for an invoice that was already settled.
I hadn't written any agent-side logic for “ignore invoices paid through other channels” — it fell out of the fact that there was only one path from “something happened” to “the agent knows about it,” regardless of which part of the system noticed it first.
That's the part that convinced me the backend design mattered as much as the memory design: an agent is only as reliable as the pipeline that decides what counts as an event worth remembering.
05
How it fits together
None of this lives in the recommendation logic. It sits underneath it — the schema, the sync job, and the one shared API that both the dashboard and the agent read from, so nothing downstream ever has to guess which version of a customer's history is current.
.
06
What I'd rebuild from day one
● Design idempotency in from the start. I added the synced_to_memory flag only after seeing a duplicate memory in testing for the second time, and rebuilding around it after the fact cost more time than starting with it would have.
● Don't sync in-flight events. My first attempt wrote a memory the moment a reminder was sent and then tried to update it once a response came in, but Hindsight memories aren't really designed to be mutated like a database row — writing once, after the outcome is known, produced far cleaner recall.
● One API beats two parallel fetches. Anywhere the dashboard and the agent both need the same underlying data, parallel fetch paths eventually drift — and when they drift, the agent's recommendation and the human's dashboard can quietly disagree with each other, which is worse than either being wrong on its own.
● Commit per row, not per batch. Batch sync jobs needed per-row commits rather than per-batch ones, so a crash mid-batch wouldn't lose progress or risk double-writing.
Most of the bugs that actually affected the agent's behavior weren't in the recommendation logic at all — they were in the pipeline deciding what became a memory and when, which is a boundary that deserved more attention early on than I initially gave it. If you're building an agent whose intelligence depends on remembering real outcomes, the backend that decides what “real” and “final” mean is not a supporting detail. It's the thing that determines whether the memory layer is trustworthy at all.
RESOURCES
Hindsight on GitHub →
Hindsight Documentation →
Agent Memory — Vectorize →