{"slug": "your-llm-extracted-10000-numbers-which-ones-are-wrong", "title": "Your LLM Extracted 10,000 Numbers. Which Ones Are Wrong?", "summary": "A study of 535 real scanned receipts from the ICDAR-2019 SROIE dataset found that metamorphic testing — checking whether an LLM returns consistent totals when the same receipt is presented in different formats — can flag extraction errors without any labeled data, but the technique's practical utility depends on how often a flagged violation corresponds to an actual mistake. The author measured that when the test fires, the output is genuinely wrong in a significant fraction of cases, though the exact rate varies by the metamorphic relation used.", "body_md": "You point an LLM at 10,000 documents and ask it to pull out a number.\n\nIt returns 10,000 numbers. Every one of them confident. Some of them wrong.\n\nYou don’t know which ones. And you can’t check by hand, because checking by hand is the entire thing you automated away.\n\nThis is the most uncomfortable fact about shipping LLM extraction in production, and most teams handle it the same way: they don’t think about it too hard. They spot-check twenty documents, the twenty look fine, and the pipeline goes live.\n\nThe textbook answer is to build a labelled test set. Pay humans to annotate a few thousand documents, measure accuracy, ship with a number. That works. It also costs weeks and real money, it goes stale the moment your input distribution shifts, and it tells you nothing about the weird new document arriving tomorrow.\n\nThere’s an older idea that sidesteps this entirely. I wanted to know whether it survives contact with real, messy data — not whether it *fires*, but whether **what it fires on is genuinely wrong**.\n\nSo I ran it on 535 real scanned receipts and measured exactly that.\n\nAsk someone the same question three different ways. If the story changes, they’re unreliable — and notice that you worked that out without knowing what actually happened.\n\nYou can do this to a model.\n\nTake a receipt. Shuffle the order of the lines. Add a “THANK YOU, PLEASE COME AGAIN” footer. Strip the currency symbols. None of that changes what the customer actually paid.\n\nRun your extractor on each version.\n\n**If it returns two different totals for what is plainly the same receipt, it just caught itself.** No answer key. No labelling budget. No reference data of any kind.\n\nThis is **metamorphic testing**, a technique built for exactly this situation — formally, the *oracle problem*: you can’t tell whether a single output is correct, so instead you assert **relations** that must hold between outputs when the input changes in a known way.\n\nEvery one of those is checkable with zero ground truth.\n\n**This is not a new idea, and I want to be upfront about that**, because the interesting part of what follows isn’t the technique. Hyun et al. built METAL, a metamorphic testing framework for LLMs. Cho, Ruberto and Terragni went further: they reviewed 1,024 papers, catalogued 191 distinct metamorphic relations across 24 NLP tasks, and released LLMorph, which implements 36 of them. Their evaluation ran over 561,000 test executions against GPT-4, Llama3 and Hermes 2, finding an average failure rate of 18%. Giskard, a commercial ML-testing company, markets the approach too.\n\nSo the method is established and the tooling exists. Here’s the question I couldn’t find anyone answering:\n\n**When one of these tests fires, how often is the output actually wrong?**\n\nFailure rates tell you how often a relation is violated. They don’t tell you whether a violation *predicts a real error*. That gap matters enormously in practice — because if you’re going to hand a human a list of flagged documents to review, you need to know what fraction of that list is worth their time.\n\nThat’s the number I set out to measure.\n\nI used the **ICDAR-2019 SROIE** dataset: 626 real scanned receipts with OCR text and ground-truth totals. Public, downloadable, genuinely messy. After filtering to receipts with a clean numeric total, 535 remained.\n\nThese are harder than they sound. The average receipt carries **13 dollar-shaped numbers** — one had 49 — and exactly one is the real total. The rest are line prices, discounts, tax lines, cash tendered, change.\n\nAnd the traps are vicious:\n\n```\nSUB-TOTAL (EX)TOTAL TAXTOTAL QTY:  1.00TOTAL SALES (EXCLUDING GST) :   31.03TOTAL SALES (INCLUSIVE OF GST) :  32.70ROUNDING ADJUSTMENTTOTAL :   31.03\n```\n\nSix lines containing the word “TOTAL.” One is what the customer paid. The bottom TOTAL : line — the one that looks most authoritative — is a reprint of the *pre-tax* figure.\n\nI wrote a receipt extractor. Not a strawman: the kind of cue-matching heuristic a competent team actually ships, and the kind of thing an LLM prompt converges to on its own. Then I ran the experiment in two strictly separated stages.\n\n**Stage 1 — blind.** Look at zero correct answers. Apply the transformations. Flag every receipt where the extractor contradicts itself.\n\n**Stage 2 — audit.** *Only now* open the answer key. Not to find errors — purely to score whether the blind flags were actually wrong.\n\nThat separation is the whole experiment. If any label leaks into Stage 1, the result means nothing. (The test suite enforces it: strip the ground-truth field entirely and Stage 1 produces identical flags. Any label read would throw.)\n\nMy first run flagged **60% of receipts**. Sixty percent! Massive detection! Ship it!\n\nIt was garbage.\n\nMy shuffle was too violent. I was scattering every line independently, which destroyed the natural adjacency between a label and its value. On a receipt where TOTAL: sits directly above 32.70, ripping those apart doesn't test robustness — it destroys information any correct extractor legitimately needs.\n\nI was penalising the model for something no correct system could survive.\n\nSo I rebuilt it. Instead of shuffling all lines, permute **semantically independent blocks**: header, line items, totals section, footer. A receipt’s totals block genuinely could appear before or after its item list — block order carries no meaning. Local structure stays intact.\n\nFlags collapsed from **72 to 4**.\n\n**Sixty-eight of my seventy-two “findings” were artifacts of my own unfair test.**\n\nI thought this was my personal embarrassment until I read the LLMorph evaluation. They hand-analysed 937 metamorphic violations and found false positive rates ranging **from 0% to 70% depending on which relation was used**. Same phenomenon, measured at scale: a badly-chosen relation generates mostly noise.\n\nWhich is the lesson I’d keep if you forget everything else here:\n\nA test that any correct system would fail isn’t a bug detector. It’s a broken test.\n\nThe engine is about fifty lines of code. It’s trivial. The hard part — the *only* hard part — is knowing which transformations are genuinely harmless. That’s not software engineering. That’s domain knowledge, and there’s no shortcut to it.\n\nWhile fixing the extractor I hit something every engineer will recognise.\n\nI had a list of “trap” strings — lines containing the word TOTAL that aren’t the payable total. SUBTOTAL. TOTAL QTY. And GST (the sales tax), because tax lines are not totals.\n\nThat last one was silently destroying the correct answer on **45 of 62 failures.**\n\nWhy? Because the real total on these receipts is frequently printed as:\n\n```\nTOTAL SALES (INCLUSIVE OF GST) :   32.70\n```\n\nMy defensive filter saw GST, decided \"tax line, skip it,\" and threw away the exact line I was looking for. A guard rule, added to prevent errors, causing 73% of them.\n\nFixing that one rule took accuracy from 48% to 87%.\n\nBut then three receipts that had been *correct* broke. My new rule preferred the GST-inclusive line — and on these three, the inclusive figure was pre-rounding:\n\n```\nTOTAL INCL GST     64.13ROUNDING            0.02TOTAL              64.15   ← what was actually paid\n```\n\nThe fix isn’t a special case. It’s a magnitude rule: a stale pre-tax reprint differs from the inclusive figure by a full tax delta (1–6 currency units), while a rounded final differs by at most one rounding step (0.05). Different by an order of magnitude, cleanly separable.\n\nI mention this because it’s the difference between publishing 87% and publishing a number that quietly embarrasses you. **Check whether your fix broke something that used to work.** Three receipts is a rounding error until someone runs your code on their data.\n\nHere's where I have to be careful, because this is exactly where articles like this usually start lying.\n\nI developed my extraction rules against the first 120 receipts. So I split the results:\n\n**Ignore the dev row.** Two flags is not a result — 100% precision on n=2 is noise wearing a suit, and I'd be embarrassed to headline it. It's in the table because leaving it out would be worse.\n\n**The held-out row is the real one.** Accuracy drops from 87% to 72% on receipts I never iterated against. That 15-point gap is the honest generalisation cost, and any single blended number would have hidden it.\n\nAnd the method still works on data it never saw:\n\n**Flagged receipts were 2.7× more likely to be wrong. 73% precision — roughly three in four flags were real errors. Found with zero labels.**\n\nThat's the number I went looking for. Not \"the relation fired 18% of the time,\" but \"when it fired, three quarters of the time something was genuinely broken.\"\n\nNow the caveat that matters most:\n\n**Recall is low — about 7%.** It flagged 13 of 535 receipts and caught 10 of roughly 130 total errors. This is not a comprehensive net. It's a **high-precision spot-checker**.\n\nThat sounds like a weakness. In practice it's the useful shape. Nobody can hand-review 10,000 outputs. But a team absolutely can review the 2% that get flagged — and when three in four are genuine errors, that review is worth someone's afternoon.\n\nThe pitch isn't \"AI finds all your bugs.\" It's *\"here are the 2% worth a human's attention, and most of them are real.\"*\n\nThe survivors are the interesting part. The strongest catches were **columnar receipts** — where OCR reads the label column and the value column as separate runs of text.\n\nReceipt 026, from a stationery shop:\n\n```\nSUB-TOTAL (EX)TOTAL TAXROUNDINGTOTALCASHCHANGE144.68144.688.68-0.01153.35153.350.00\n```\n\nThe word TOTAL and the number 153.35 never appear on the same line. They're not even close together. The extractor has to know that the fourth label maps to the fifth value — and it doesn't. It returns 144.68 in one block order and 8.68 in another.\n\nTwo different answers for the same receipt. That's a contradiction, and it took no answer key to find it.\n\nThe ground truth, when I finally looked, was 153.35. Wrong *and* inconsistent — but the inconsistency is what surfaced it, blind.\n\n**And one it got wrong.** Receipt 225 was flagged, but the base extraction was correct — the extractor happened to land on the right number and was merely fragile about it. That's a false positive, it's why precision is 73% and not higher, and it's visible in the tool's own output. If it weren't there, you should trust the other numbers less.\n\nI packaged the engine as **wobbly** — because that's what it detects. A system that gives different answers to the same question is wobbly, and wobbly is a bug even when the answer happens to land right.\n\n```\npip install wobbly\n```\n\nCode, data, and the full experiment: **github.com/tarunagarwal1981/wobbly**\n\nIt knows nothing about receipts and never looks inside your system. You give it two things:\n\nHere's a complete, runnable example — a deliberately naive extractor that takes the last \"total\"-ish line it sees, which makes it order-dependent:\n\n``` python\nimport randomfrom wobbly import check, Relation, unchangeddef extract_total(receipt):    total = None    for line in receipt[\"lines\"]:        if \"total\" in line.lower():            amounts = [w for w in line.split() if w.replace(\".\", \"\").isdigit()]            if amounts:                total = float(amounts[-1])    return totalrng = random.Random(0)def shuffle_lines(receipt):    lines = list(receipt[\"lines\"])    rng.shuffle(lines)    return {\"lines\": lines}reorder = Relation(    name=\"reorder lines => total unchanged\",    transform=shuffle_lines,    assertion=unchanged(),)receipt = {\"lines\": [    \"Flat White     4.50\",    \"Muffin         3.00\",    \"SUBTOTAL       7.50\",    \"TOTAL          7.95\",    \"CASH          10.00\",]}report = check(extract_total, receipt, [reorder], samples=20)print(report.summary())\n```\n\nOutput:\n\n``` js\nBROKE (1 of 5 trials):  [reorder lines => total unchanged] expected 7.95 to be preserved, got 7.5\n```\n\nIt found the bug in five trials. The extractor picks up SUBTOTAL when the lines arrive in a different order — and nothing in that code ever knew the right answer was 7.95.\n\nThere's a built-in pack too, if you're working with receipt-shaped documents:\n\n``` python\nfrom wobbly import check, default_packreport = check(lambda r: extract_total(r), receipt, default_pack())\n```\n\n**One contract to know:** base_input, your system, and every transform must all agree on the same input shape, because checkfeeds each transformed input straight back into your system.\n\nCrucially, **it modifies the input before your code ever sees it.** Nothing is injected into your prompt. Your function receives what looks like an ordinary document and does what it always does. That's why it works on anything — LangChain pipeline, raw API call, fine-tuned model, regex parser. It doesn't care.\n\nSwap receipts for support tickets, contracts, or lab reports and the same code works unchanged. What changes is the list of harmless transformations — and that list is where your domain expertise lives.\n\nIf you want breadth of pre-built relations rather than a minimal engine, use LLMorph — 36 relations across four NLP tasks, plus a catalogue of 191 more in their paper. I built something smaller because I wanted to test *document extraction pipelines*, not NLP benchmarks, and I wanted the whole thing auditable in one sitting.\n\n**One honest cost:** each relation costs at least one extra call to your system. Randomised relations (like reordering) run several times to explore different permutations; deterministic ones (adding a footer, stripping currency symbols) run once. You wouldn't run this on every production document. Run it in CI on 50 representative documents whenever you change a prompt or a model version, or sample 2% in production. It's a testing cost, not a runtime cost — the same way you don't run your unit tests on every user request.\n\nThe boundary matters more than the technique.\n\n**If you can compute the right answer, compute it.** If a field must be a valid date, check it's a valid date. If line items must sum to the total, sum them. If physics gives you an expected value, use the physics. Metamorphic testing is strictly worse than a direct check — it only tells you the system disagreed with itself, not what the right answer was.\n\nSelf-consistency is what you reach for when **no formula exists**. Which, for anything involving reading a document, is most of the time. There is no equation that tells you whether an LLM read a scanned receipt correctly. The only ground truth is the paper, and if you had someone checking every piece of paper you wouldn't need the model.\n\nThat's the gap this fills — and only that gap.\n\n**Metamorphic flags predict real errors — but only some of them.** 2.7× enrichment, 73% precision, 7% recall, on held-out data. High-precision spot-checking, not comprehensive coverage. That's a useful tool and a modest claim, and I'd rather publish the modest version.\n\n**Your test is a hypothesis too.** My first version was 94% wrong and looked like a triumph. The academic work found false positive rates up to 70% depending on the relation. Sanity-check the test before you trust its output.\n\n**The engine is the easy part.** Fifty lines. What's hard is knowing which changes are genuinely harmless in your domain — and that's exactly the thing you already know and a generic tool never will.\n\n*Everything here is reproducible:*\n\n```\ngit clone https://github.com/tarunagarwal1981/wobblycd wobblypython scripts/run_blind.py\n```\n\n*No API key, no network — you get the exact numbers in that table. **pip install -e \".[test]\" && pytest runs the suite, including a test that strips the ground-truth field entirely and confirms the blind stage produces identical flags. A second script rebuilds the dataset from the original ICDAR source and asserts byte-identity against the committed slice, so you can verify I didn't cherry-pick which receipts to include.*\n\n**References**\n\n*What invariants hold in your domain? I'm collecting the ones that actually catch production bugs — if you've got one, I'd like to hear it.*\n\n[Your LLM Extracted 10,000 Numbers. Which Ones Are Wrong?](https://pub.towardsai.net/your-llm-extracted-10-000-numbers-which-ones-are-wrong-7a5d54050dd3) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/your-llm-extracted-10000-numbers-which-ones-are-wrong", "canonical_source": "https://pub.towardsai.net/your-llm-extracted-10-000-numbers-which-ones-are-wrong-7a5d54050dd3?source=rss----98111c9905da---4", "published_at": "2026-07-20 14:01:05+00:00", "updated_at": "2026-07-20 14:43:17.047723+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools"], "entities": ["ICDAR-2019 SROIE", "METAL", "LLMorph", "Giskard", "GPT-4", "Llama3", "Hermes 2"], "alternates": {"html": "https://wpnews.pro/news/your-llm-extracted-10000-numbers-which-ones-are-wrong", "markdown": "https://wpnews.pro/news/your-llm-extracted-10000-numbers-which-ones-are-wrong.md", "text": "https://wpnews.pro/news/your-llm-extracted-10000-numbers-which-ones-are-wrong.txt", "jsonld": "https://wpnews.pro/news/your-llm-extracted-10000-numbers-which-ones-are-wrong.jsonld"}}