An agent-native product isn't a chatbot bolted onto a CRUD app — it's an MCP server that lets an agent do things. Read invoices, create expenses, mark a client overdue. The demo is easy. The part that decides whether the thing survives contact with real traffic is the layer nobody screenshots: what happens when a write is sent twice.
This is the unglamorous half of agent-native engineering. Every write operation an agent can invoke needs idempotency, safe retries, and typed recoverable errors — not as a nice-to-have, but as the difference between a product and a liability. Here's why, and how to build it.
Distributed systems people have known forever that networks give you at-least-once delivery, not exactly-once. A request goes out, the response gets lost on the way back, the caller doesn't know if the write landed, so it retries. With a human clicking a button, this is rare enough to hand-wave.
Agents remove the hand-wave. Three things about how agents call tools push duplicate writes from "edge case" to "Tuesday":
429
and 5xx
. The Frihet MCP server (fri_
key, and its own client retries with exponential backoff when it hits that ceiling. A retry after a 429
is MCP itself does not save you here. A tool call is just RPC over JSON — the protocol says nothing about whether invoking create_invoice
twice creates one invoice or two. That contract is entirely yours to define on the server. If you don't define it, you've defined it as "two."
For a read tool, a duplicate is free. For a write tool in a fiscal domain — an ERP that prepares VeriFactu records, IVA and IRPF filings — a duplicate is a phantom invoice in someone's tax quarter. The blast radius is exactly why the boring layer matters.
The fix is to let the caller assign a stable identity to a logical operation, independent of how many times it's physically delivered. That's an idempotency key: a client-generated token (usually a UUID) that stays the same across retries of the same intended write, and changes for a genuinely new one.
The rule for the client is simple: generate the key once, at the point of intent, and reuse it for every retry of that same intent. If the agent decides to bill a client, it mints one key and carries it through all four delivery attempts. When it decides to bill a different client, it mints a new one.
The server's job is to make the key mean something: the first time it sees a key, do the work and remember the result; every subsequent time, return the remembered result without touching the database again.
The subtlety that trips people up is atomicity. "Check if the key exists, then insert it" is a race — two concurrent retries both read "not found" and both do the work. The claim has to be a single atomic operation, which in Postgres is an INSERT ... ON CONFLICT DO NOTHING
and in most KV stores a compare-and-set.
import { createHash } from "node:crypto";
interface WriteContext {
accountId: string; // scope keys per tenant — never globally
toolName: string; // e.g. "create_invoice"
idempotencyKey: string; // client-supplied, stable across retries
}
async function runOnce<T>(
ctx: WriteContext,
payload: unknown,
work: () => Promise<T>,
): Promise<T> {
const slot = `${ctx.accountId}:${ctx.toolName}:${ctx.idempotencyKey}`;
const fingerprint = createHash("sha256")
.update(JSON.stringify(payload))
.digest("hex");
// Atomic claim: only the FIRST caller inserts the row.
const claim = await store.claim(slot, fingerprint); // INSERT ... ON CONFLICT DO NOTHING
if (!claim.inserted) {
// Key already seen. Same body -> return the stored result.
// Different body -> the client reused a key for a new request. Reject.
if (claim.fingerprint !== fingerprint) {
throw new ToolError("idempotency_key_reused", {
retryable: false,
message: "Idempotency key was reused with a different payload.",
});
}
return store.awaitResult<T>(slot); // return the SAME result, never re-run
}
try {
const result = await work();
await store.complete(slot, result);
return result;
} catch (err) {
await store.fail(slot); // release the slot so a genuine retry can proceed
throw err;
}
}
Three things this snippet gets right that naive versions miss:
accountId
in the slot, one customer's retry could return another's invoice. Scope aggressively.How long do you keep the record? Long enough to outlive any retry storm — hours to a day is typical, and it's a deliberate trade-off, not a default. Too short and a slow retry slips through the window and double-writes; too long and you're storing every request body forever.
try/catch
Idempotency makes retrying safe. It doesn't tell you when to retry. That decision depends entirely on what went wrong, and "an error happened" is not enough information to make it.
429
or 503
is transient — back off and retry.422 validation_failed
is permanent — retrying the same bad payload a thousand times just burns your rate limit against a wall.504
gateway timeout is the genuinely hard one: the write For the agent to make this call, the error has to be machine-readable. A stringified 500 Internal Server Error
forces the model to guess from prose, and models guess wrong. Ship a typed error taxonomy instead: a stable code
, an explicit retryable
boolean, and a hint for how long to wait.
type ErrorCode =
| "rate_limited" // 429 — back off, then retry
| "upstream_timeout" // 504 — retry with the SAME idempotency key
| "validation_failed" // 422 — do NOT retry; the payload is wrong
| "insufficient_funds" // 402 — do NOT retry; surface to the user
| "idempotency_key_reused"// — client bug; do NOT retry
| "conflict"; // 409 — a concurrent write won; re-read, then decide
class ToolError extends Error {
constructor(
public readonly code: ErrorCode,
public readonly meta: {
retryable: boolean;
message: string;
retryAfterMs?: number;
},
) {
super(meta.message);
}
}
// The client/agent reads `retryable` — it never parses a stringified 500.
function isRetryable(err: unknown): err is ToolError {
return err instanceof ToolError && err.meta.retryable;
}
Pair the taxonomy with exponential backoff and jitter. Backoff without jitter means every one of your fanned-out tool calls that got rate-limited retries at the same instant — a synchronized stampede that trips the limit again. Randomizing the delay spreads the herd out. And cap the attempts: retryable does not mean retry forever.
Note that retryable
lives on the server's side of the contract but is consumed by the client. That's the point — the server knows whether a write is safe to repeat, so it says so explicitly instead of making every caller reverse-engineer the answer from an HTTP status and a hope.
A demo proves an agent can create an invoice. Production asks a harder question: when the same instruction gets delivered twice — because a network blipped, a session resumed, or a 429
triggered the retry the client was designed to perform — does your customer end up with one invoice or two?
Consider the surface area. A server like Frihet exposes 157 tools over mcp.frihet.io
, a large share of them writes into an ERP that generates real tax records. Every one of those write tools is a place where at-least-once delivery meets a fiscal side effect. Idempotency keys, atomic server-side dedup, and typed recoverable errors are what turn that surface from a duplicate-data generator into something an agent can hammer safely at 100 requests a minute. That contract is the argument of this article: it is what every server on the other end of an agent owes its callers. It is also the least glamorous work in the codebase and the work that determines whether the product is trustworthy.
The industry spent a decade learning this for payments — it's why Stripe made idempotency keys a first-class header. Agents make the lesson urgent everywhere at once, because agents retry, resume, and fan out by default. If you're building anything an agent can write to, treat this as the foundation, not the polish. The chatbot is the part users see. This is the part that keeps their data correct.
Next up: designing tool schemas an agent won't misuse — types, enums, and error contracts as the real interface.