{"slug": "dead-letter-replay-does-not-belong-on-free-inference", "title": "Dead-Letter Replay Does Not Belong on Free Inference", "summary": "A developer argues that dead-letter queue replay should be governed by deterministic code rather than free inference, since replaying a failed message can re-trigger payments, webhooks, inventory mutations, or email sends. The proposed Python replay gate reads durable fields such as topic, error class, attempt count, and idempotency key, returning one of three verbs — never, replay_once, or hold — and defaults unknown cases to hold so new error codes fail closed. The model, if used at all, only writes a proposal file and never calls the broker.", "body_md": "Live replay of a dead-letter message is a control-plane act. Free inference should not own it.\n\nA dead-letter queue is not a suggestion box. It is a holding pen for work that already failed, often after a side effect has been attempted. Replay means the consumer will call a payment API, emit a webhook, mutate inventory, or send mail again. The payload may be poison. It may be a duplicate of a request that already committed. It may be a timeout whose remote party succeeded while the local waiter expired. Those cases do not share a prompt. They share a contract, and the contract has to survive a model outage.\n\nTeams reach for a chat completion because the dump looks linguistic. Error strings wander. Partner codes collide. A model can cluster the mess into a tidy story. That clustering is useful on a desk. It is a poor runtime for a switch that re-enters the world.\n\nThink of the DLQ as a loaded elevator. Classification is the inspection tag on the door. Replay is the button that sends the car. Handing the button to a volunteer intern with a sticky note is the free-inference pattern: cheap, fluent, and unbound to the interlock.\n\nThe failure mode is not only a wrong label. Free inference is a best-effort path. It can stall, rate-limit, or answer with a different JSON shape than yesterday. A consumer that blocks on that path turns a message bus into a chat session. A consumer that proceeds on timeout turns a maybe into a charge. Neither is an SLO.\n\nA replay gate belongs in deterministic code next to the consumer. The gate reads durable fields: topic, error class, attempt count, idempotency key, and a human-committed policy table. It returns one of three verbs. `never` parks the message for a ticket. `replay_once` requeues with a fresh attempt counter and the same key. `hold` waits for an operator. The model, if it appears at all, writes a proposal file. It does not call the broker.\n\nThe artifact below is that gate. It is ordinary Python. It is meant to be copied into a worker repo and tested without a network.\n\n``` python\n# replay_gate.py\nfrom dataclasses import dataclass\nfrom enum import Enum\nfrom typing import Mapping, Optional, Tuple\n\nclass Verb(str, Enum):\n    NEVER = \"never\"\n    REPLAY_ONCE = \"replay_once\"\n    HOLD = \"hold\"\n\n@dataclass(frozen=True)\nclass DeadLetter:\n    topic: str\n    error_class: str\n    attempts: int\n    idempotency_key: str\n    body_hash: str\n    already_committed: bool\n\nPolicyKey = Tuple[str, str]\nPOLICY: Mapping[PolicyKey, Verb] = {\n    (\"payment.capture\", \"timeout\"): Verb.REPLAY_ONCE,\n    (\"payment.capture\", \"insufficient_funds\"): Verb.NEVER,\n    (\"payment.capture\", \"duplicate\"): Verb.NEVER,\n    (\"inventory.reserve\", \"lock_timeout\"): Verb.REPLAY_ONCE,\n    (\"inventory.reserve\", \"overbook\"): Verb.NEVER,\n    (\"webhook.partner\", \"http_503\"): Verb.REPLAY_ONCE,\n    (\"webhook.partner\", \"http_400\"): Verb.NEVER,\n    (\"mail.receipt\", \"smtp_421\"): Verb.REPLAY_ONCE,\n    (\"mail.receipt\", \"unknown_recipient\"): Verb.NEVER,\n}\n\nMAX_ATTEMPTS = 3\n\ndef decide(letter: DeadLetter, policy: Mapping[PolicyKey, Verb] = POLICY) -> Verb:\n    if not letter.idempotency_key:\n        return Verb.HOLD\n    if letter.already_committed:\n        return Verb.NEVER\n    if letter.attempts >= MAX_ATTEMPTS:\n        return Verb.HOLD\n    verb = policy.get((letter.topic, letter.error_class))\n    if verb is None:\n        return Verb.HOLD\n    return verb\n\ndef apply(letter: DeadLetter, broker, audit) -> Verb:\n    verb = decide(letter)\n    audit.record(letter.body_hash, verb.value)\n    if verb is Verb.REPLAY_ONCE:\n        broker.requeue(letter, extra_headers={\"x-replay\": \"1\"})\n    return verb\n```\n\nThe unknown row is the whole point. A missing `(topic, error_class)` pair does not become a prompt. It becomes `hold`. That is how a new partner code fails closed instead of becoming a creative retry. The `already_committed` bit is equally blunt. If the outbox, the payment ledger, or the partner receipt says the work landed, replay is vandalism dressed as healing.\n\nTests pin the verbs. They do not pin a temperature.\n\n``` python\n# test_replay_gate.py\nfrom replay_gate import DeadLetter, Verb, decide\n\ndef letter(**kwargs):\n    base = dict(\n        topic=\"payment.capture\",\n        error_class=\"timeout\",\n        attempts=1,\n        idempotency_key=\"cap_9f3a\",\n        body_hash=\"sha256:abc\",\n        already_committed=False,\n    )\n    base.update(kwargs)\n    return DeadLetter(**base)\n\ndef test_timeout_replays_once():\n    assert decide(letter()) is Verb.REPLAY_ONCE\n\ndef test_committed_work_never_replays():\n    assert decide(letter(already_committed=True)) is Verb.NEVER\n\ndef test_unknown_error_holds():\n    assert decide(letter(error_class=\"weird_partner_code\")) is Verb.HOLD\n\ndef test_missing_idempotency_key_holds():\n    assert decide(letter(idempotency_key=\"\")) is Verb.HOLD\n\ndef test_insufficient_funds_never_replays():\n    assert decide(letter(error_class=\"insufficient_funds\")) is Verb.NEVER\n```\n\nRun them with `python -m pytest test_replay_gate.py -q`. The suite is the oracle. A model that restates the same rules in prose is commentary, not control.\n\nFree inference still has a desk job. Unstructured DLQ dumps are noisy, and a first pass that clusters raw error strings into candidate `(topic, error_class)` rows can save an afternoon. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A workspace with free model access and a free server option can sit beside those dumps, emit a proposed policy patch, and stop there. The patch lands in review like any other config change. The consumer never calls the model on the hot path.\n\nThat rehearsal looks like a file, not a tool call inside `apply`.\n\n```\n# propose_policy.py — offline sketch, not a worker dependency\n# Label as unexecuted until a human copies rows into POLICY.\nPROPOSAL_SCHEMA = {\n    \"type\": \"object\",\n    \"required\": [\"topic\", \"error_class\", \"verb\", \"rationale\"],\n    \"properties\": {\n        \"topic\": {\"type\": \"string\"},\n        \"error_class\": {\"type\": \"string\"},\n        \"verb\": {\"enum\": [\"never\", \"replay_once\", \"hold\"]},\n        \"rationale\": {\"type\": \"string\"},\n    },\n}\n\ndef write_proposal(path, rows):\n    # Persist proposals for review. Do not import this module from the consumer.\n    import json\n    from pathlib import Path\n    Path(path).write_text(json.dumps(rows, indent=2), encoding=\"utf-8\")\n```\n\nRed flags show up before the first wrong charge. The consumer imports an SDK in the same process that acknowledges Kafka or SQS. Replay waits on a completion token. Two identical payloads, five minutes apart, receive two verbs. The policy lives only in a system prompt. The audit log stores the model's paragraph and not the verb. Any one of those is enough to pull the model out of the loop.\n\nBetter alternatives are boring on purpose. Map partner error codes at the edge. Persist an idempotency key before the first attempt, not after the failure. Record `already_committed` from the ledger, not from a summary of logs. Cap attempts in the broker. Put unknown classes on `hold` and page a human. If the dump is truly unstructured, run the clusterer on a sample in a scratch environment, then type the surviving rows into `POLICY`.\n\nExit criteria belong next to the policy, not in a runbook nobody opens. Stop using even the offline proposer when the same payload yields two different verbs across sessions. Stop when proposal latency exceeds the time a reviewer would spend reading twenty raw lines. Stop when the topic moves money, identity, or medical data through a third party that is not under the team's data agreement. Stop when the DLQ volume is a load test in disguise: a replay storm is a traffic generator, and a chat API is not a traffic shaper.\n\nWho should not use this approach is as important as the gate. A team without idempotency keys should not replay at all, with or without a model. A team whose ledger cannot answer `already_committed` should park every payment topic on `hold`. A team that needs sub-second classification under partition loss should not add a network hop that is allowed to vanish. A regulated workload that cannot put payloads on a shared free endpoint should keep the dumps inside the boundary and type the table by hand.\n\nLimitations of the gate itself are real. The table lags a new partner. `error_class` is only as good as the parser that produces it. `MAX_ATTEMPTS` does not know about downstream quotas. `HOLD` can hide a growing pile if nobody pages. The design accepts those limits because they are visible. A fluent paragraph is not visible in the same way. It fails like weather.\n\nThe current wave of agent demos makes the anti-pattern tempting. Tool calling looks like a universal adapter: feed the DLQ body in, get a verb out, let the agent requeue. That adapter is a completion. Completions are not leases, not locks, and not ledgers. They are a way to draft the table that the lease, the lock, and the ledger already require.\n\nA free model and a free server are enough to rehearse that table against fixtures. They are not a substitute for the switch. Keep the intern at the desk. Keep the elevator button in code.", "url": "https://wpnews.pro/news/dead-letter-replay-does-not-belong-on-free-inference", "canonical_source": "https://dev.to/aiio_6471/dead-letter-replay-does-not-belong-on-free-inference-4jgo", "published_at": "2026-09-23 17:21:12+00:00", "updated_at": "2026-09-23 17:29:06.692438+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/dead-letter-replay-does-not-belong-on-free-inference", "markdown": "https://wpnews.pro/news/dead-letter-replay-does-not-belong-on-free-inference.md", "text": "https://wpnews.pro/news/dead-letter-replay-does-not-belong-on-free-inference.txt", "jsonld": "https://wpnews.pro/news/dead-letter-replay-does-not-belong-on-free-inference.jsonld"}}