# Dead-Letter Queues for LLM Extraction Failures: Capture, Triage, and Replay Without Losing Trust > Source: > Published: 2026-07-24 06:20:16+00:00 A validation failure is not an exception to hide. It is a record your system does not yet know how to trust. That distinction matters in LLM extraction pipelines. A malformed invoice, an unexpected OCR layout, a model response that violates the schema, and a semantically impossible value may all reach the same line of validation code. If the only outcomes are “retry” or “drop,” the pipeline will either waste money repeating the same failure or silently lose work. The production answer is a dead-letter path: a durable place for failed records to wait with enough evidence to explain, triage, and safely replay them. The queue itself is the easy part. The hard part is designing the failure contract around it. [Constrained decoding and post-hoc validation solve different problems](https://dev.to/blog/constrained-decoding-vs-post-hoc-validation-llm-extraction/). Even with both, some records should fail. Real documents are messy, schemas change, OCR corrupts values, and models sometimes return plausible nonsense. A robust validation boundary should produce more than `true` or `false` . It should emit a reason the rest of the pipeline can act on: That result becomes a routing decision. High-confidence, valid records can flow forward. Recoverable transport failures can use a bounded retry. Ambiguous or invalid records belong in review or a dead-letter queue. [Confidence-based routing](https://dev.to/blog/confidence-scores-llm-extraction/) is useful precisely because “trust everything” and “review everything” are both bad operating models. Putting the original input on another queue is not enough. Without context, the team investigating the failure has to reconstruct the run from scattered logs—if those logs still exist. I would store a dead-letter envelope containing: This is less about collecting every possible field and more about preserving the decision. Six days later, an engineer or reviewer should be able to answer: what did the system see, what did it produce, which contract rejected it, and can it be replayed safely? Do not turn the dead-letter queue into a shadow database. Store references when the source of truth already exists, define retention and deletion rules, and avoid copying sensitive document contents into systems with weaker controls. Not every failure deserves another model call. I separate failures into a few broad classes: The retry policy should be based on that classification. [Retry budgets and backpressure](https://dev.to/blog/backpressure-retry-budgets-llm-services/) matter because a provider incident can otherwise turn one failed request into a storm of expensive duplicates. A dead-letter queue is only useful if records can leave it safely. The dangerous replay implementation simply sends the record back to the start. If earlier attempts already wrote partial state, emitted events, or triggered downstream actions, replay can create duplicate invoices, duplicate notifications, or inconsistent audit trails. Safe replay needs an idempotency boundary. Give each logical extraction a stable key. Make downstream writes upsert or compare against a known processing version. Record which stages completed. Re-run only the stages affected by the fix when possible. I also want replay to name the change that justifies it: “Try again” is not a remediation strategy. “Replay against schema v4 after fixing the currency parser” is. Human-in-the-loop systems often fail in a quieter way: they create a review screen, ask an operator to correct a value, and throw away the reason. A useful review workflow captures structured outcomes: Those outcomes improve more than the one record. They reveal recurring document formats, brittle fields, bad thresholds, and failure cohorts worth fixing upstream. They can also become curated evaluation examples, provided privacy and data-governance rules allow it. The goal is not to keep humans in the loop forever. It is to spend human attention where risk is high and turn repeated review work into engineering feedback. A DLQ with no metrics is an archive of surprises. At minimum, I would track: Watch the rate, not only the count. Traffic growth can make the raw count rise while reliability improves. A sudden failure-rate spike after a schema, prompt, OCR, or model change is a much cleaner drift signal. This connects the dead-letter path to the broader observability story: traces explain an individual failure; aggregate metrics show whether the system is becoming less trustworthy. [Schema drift](https://dev.to/blog/schema-drift-llm-pipelines/) often appears first as a change in validation failures by field or document cohort. The architecture I keep coming back to is: This is the operational layer around tools such as [ confident-extract](https://pypi.org/project/confident-extract/). The library is published on PyPI and focuses on deterministic structured extraction, validation, and confidence. The queueing, review, retention, and replay design belong to the application around that boundary. That separation is important. A useful open-source component should make its boundary sharper, not claim to be the entire production system. Teams often design the happy path first and treat failed records as an operations problem to solve later. In an LLM system, the failure path is part of the normal path. Probabilistic components, messy source data, and changing contracts guarantee that some records will need a different decision. The system earns trust by making that decision explicit, durable, observable, and reversible. Do not drop the record. Do not retry it forever. Preserve the evidence, route it by risk, and make replay a controlled engineering action. That is what turns “the model failed” from an incident into a workflow. *Written by Hitarth Desai ( hitarthbuilds), an AI Systems Engineer building reliable LLM extraction and MLOps pipelines. His open-source confident-extract package is available on PyPI. promptcrucible remains in active development.* Originally published by [Hitarth Desai](https://hitarthdesai.com/) (`hitarthbuilds` ) at [https://hitarthdesai.com/blog/dead-letter-queues-llm-extraction-pipelines/](https://hitarthdesai.com/blog/dead-letter-queues-llm-extraction-pipelines/)