> TL;DR — I open-sourced an AWS sample that makes LLM document extraction learn from human corrections. Each QA fix is captured once and reused, so the pipeline gets more accurate and cheaper over time — with no retraining and no redeployment. Repo: github.com/aws-samples/sample-prompt-correction-memory. You can run the whole self-healing loop locally in 30 seconds, no AWS account needed.
If you've built an LLM extraction pipeline, you know this pain.
Your QA analyst opens today's invoice. The model extracted payment_terms as "quarterly." That's wrong — "quarterly" is the billing frequency; the actual payment term is Net 30. The analyst corrects it. Done.
Tomorrow, an identical invoice from the same vendor arrives. The model extracts... "quarterly" again. Same mistake. The analyst corrects it again.
The correction evaporated. It fixed one document and taught the system nothing.
Multiply that across thousands of documents and dozens of recurring error patterns, and you get the two options most teams settle for:
There's a third way that needs neither.
What if every human correction became a permanent, reusable asset?
That's the pattern behind sample-prompt-correction-memory. Instead of throwing corrections away, it stores them and feeds them back into future extractions — so the same mistake is never made twice.
It works as three tiers, and the magic is that traffic gradually shifts from the expensive tier to the free one:
When the same correction pattern recurs enough times, the system synthesizes a deterministic rule (a regex, a lookup, a normalization) that handles that field with zero LLM cost and sub-millisecond latency. "Net 30" → 30 becomes a rule; it never needs a model again.
Fields with no matching rule go to Amazon Bedrock (Claude Haiku 4.5). Every field comes back with a confidence score. If confidence clears the threshold, accept it and move on.
If confidence is below threshold, the system retrieves the most relevant past corrections, formats them as few-shot examples, and re-extracts with a stronger model (** Claude Sonnet 4.5**). The model learns from the specific mistakes it made before — in-context, no training.
Every QA correction flows into a correction log (DynamoDB) that powers both the self-healing retrieval and the rule graduation. So the more the system is used, the smarter and cheaper it gets.
The clearest fit is high-repetition, back-office document processing — accounts payable, contracts, compliance filings, supply-chain docs — where the same field errors recur and analyst time is the real expense.
Fully serverless, deployed with AWS SAM:
Document ──► S3 ──► EventBridge ──► SQS ──► Lambda (extract)
│
┌───────────────────┼───────────────────┐
▼ ▼ ▼
Tier 1: Rules Tier 2: Bedrock Tier 3: Self-heal
(deterministic) (Claude Haiku 4.5) (Claude Sonnet 4.5)
▲ │
│ ▼
└──────── Correction Log (DynamoDB) ◄────┘
(grows from QA feedback)
QA corrections are uploaded as JSON to an S3 corrections/ prefix; a second Lambda validates and ingests them into the correction log. Everything is encrypted with a customer-managed KMS key, and IAM is scoped to the specific Bedrock model ARNs the sample uses.
The quickstart runs the entire self-healing loop with a mocked Bedrock client, so you can watch the mechanism without deploying anything:
git clone https://github.com/aws-samples/sample-prompt-correction-memory
cd sample-prompt-correction-memory
pip install -e ".[dev]"
python examples/quickstart.py
You'll see output like this:
Step 1: Initial extraction (no correction memory)
Field: effective_date
Value: March 2024 Confidence: 0.55 ❌ NO
Step 2: Self-healing triggers (0.55 < threshold 0.70)
Retrieving corrections for 'effective_date'...
Found: "March 2024" → "2024-03-01"
Step 3: Re-extraction with correction memory
Value: 2024-03-01 Confidence: 0.95 ✓ YES
Self-Healed: True
Low-confidence extraction → retrieve past correction → re-extract → correct answer. No retraining. No redeployment.
With the AWS CLI, SAM CLI, and Bedrock model access for Claude Haiku 4.5 + Sonnet 4.5:
make deploy # S3, DynamoDB, Lambda, EventBridge, SQS, KMS
make seed # load sample corrections into the correction log
make trigger # upload a sample document → triggers real extraction
make verify # read back results (fields, confidence, self_healed)
make destroy # empty buckets + delete the stack
make verify prints each extracted field with its confidence and whether self-healing kicked in — your proof it works end-to-end.
This sample provides semantic correction memory (what was extracted wrong, and why). Its companion, sample-textract-field-memory, provides spatial memory (where fields appear on a document layout). Together they form a dual memory for document pipelines: use the cheap spatial lookup when you're confident where a field is, and fall back to self-healing extraction when you're not.
Because it's more useful when it's used well:
The repo is open source under aws-samples, MIT-0 licensed, with a benchmark suite, an interactive dashboard, and an offline test harness.
👉 github.com/aws-samples/sample-prompt-correction-memory
What repetitive extraction error is your team still fixing by hand? I'd love to hear which patterns you'd want a system like this to learn.
This is an open-source AWS sample intended for demonstration and non-production use.