{"slug": "i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying", "title": "I built an AI-first bank statement converter. The one rule that stopped it lying to me.", "summary": "A developer built StatementDecoder, an AI-first bank statement converter, and discovered that the hardest part was not reading statements but verifying the output. After encountering errors from diverse formats like UK bank PDFs, Wise statements, and Revolut CSVs, the developer added an independent validation step using balance reconciliation, ensuring errors are caught before export. The project highlights that LLMs don't need to be perfect if outputs can be verified independently.", "body_md": "When I started building [StatementDecoder](https://statementdecoder.com), I thought the difficult part would be reading bank statements.\n\nIt wasn't.\n\nReading them turned out to be relatively easy.\n\nThe difficult part was knowing when the output was wrong.\n\nLike most people building with LLMs, my first version was straightforward. Feed the statement to the model and ask for structured JSON. For a while it felt almost magical. It handled tables I'd never written parsers for. It coped with different layouts. It even did a decent job on scanned PDFs.\n\nThen I stopped testing with the handful of sample statements I'd been using and started collecting real ones.\n\nEverything fell apart.\n\nA PDF from one UK bank looked nothing like another. Wise statements had quirks I'd never seen before. Revolut CSVs weren't as simple as I expected. Then there were OFX files that looked like they hadn't been touched since the late 90s.\n\nThe model still did a good job.\n\nJust not a perfect one.\n\nAnd that's the problem.\n\nImagine a 14-page statement with a few hundred transactions. The model gets every row right except one. Maybe it swaps two digits in an amount. Maybe a transaction wrapped across two pages and it guessed the wrong date. Maybe it silently merged two rows together.\n\nThe JSON still looks completely reasonable.\n\nNothing says, \"By the way, transaction 187 is wrong.\"\n\nIf you're importing the data into bookkeeping software, that one mistake is enough to make the whole export untrustworthy.\n\nMy first reaction was the obvious one: improve the prompt.\n\nThen I tried changing models.\n\nThen I spent time tweaking extraction logic.\n\nEventually I realised I was solving the wrong problem.\n\nI was asking the model to tell me the answer and trusting the same model to tell me whether the answer was correct.\n\nThat's a terrible idea.\n\nThe thing that finally fixed the architecture wasn't a better prompt. It was adding something that didn't care where the data came from.\n\nBank statements are actually quite nice because they contain their own consistency checks.\n\nThe opening balance, every transaction and the closing balance all have to agree.\n\nRunning balances have to reconcile as well.\n\nThat means every extraction, whether it came from an LLM, OCR, a CSV import or an OFX parser, can go through exactly the same validation step.\n\nIf everything adds up, great.\n\nIf it doesn't, we know something is wrong before the user ever exports the data.\n\nSometimes it's obvious what's happened. A page is missing. A debit became a credit. A row was duplicated. Sometimes the validator can repair the issue automatically. Sometimes it can't.\n\nEither way, the system doesn't pretend everything is fine.\n\nThat ended up changing how I think about building with AI.\n\nThe model doesn't need to be perfect.\n\nIt just needs to produce something that can be verified independently.\n\nSome of the bugs that led us there were surprisingly mundane.\n\nOne Wise statement looked as though the description column had been extracted perfectly. Every row contained text. Unfortunately, the text was mostly internal transaction IDs. The merchant names lived elsewhere on the page.\n\nRevolut caught me for a different reason. Their CSV exports include a **Type** column with values like `CARD_PAYMENT`\n\nand `TOPUP`\n\n. I initially assumed that was the transaction direction. It wasn't. Once I stopped trying to infer debits and credits from that field and trusted the signed amounts instead, the imports became much more reliable.\n\nUK bank statements introduced another edge case. Quite a few of them don't use negative numbers at all. Instead they have separate Money In and Money Out columns. One parser decided the mostly empty Money In column wasn't part of the table and quietly removed every transaction direction.\n\nThen there was an Australian OFX export that wrapped CDATA inside SGML. OFX 1.02 already isn't proper XML, so by the time I'd finished preprocessing it I'd written two tiny functions that completely eliminated what I'd assumed would require another AI pass.\n\nThat probably taught me the biggest lesson of the project.\n\nNot every difficult-looking problem needs AI.\n\nSometimes it just needs twenty lines of code.\n\nThe same thing happened with scanned PDFs.\n\nOriginally we'd send every PDF through the same pipeline. If the document had no text layer, we'd waste tokens asking the model to extract text that simply wasn't there before eventually failing.\n\nNow the first thing we do is check whether any text exists at all.\n\nIf it doesn't, we route the document straight to vision.\n\nIt's faster, cheaper and produces better results.\n\nLooking back, I don't think the interesting part of StatementDecoder is that it uses AI.\n\nLots of products use AI.\n\nThe interesting part is deciding where **not** to trust it.\n\nThe model is responsible for understanding messy documents.\n\nDeterministic code is responsible for parsing structured formats.\n\nThe validator is responsible for deciding whether the result is believable.\n\nThose are three separate jobs.\n\nKeeping them separate made the system dramatically more reliable than trying to make the model do everything.\n\nIf you're building any kind of extraction pipeline, not just for bank statements, that's probably the idea I'd steal.\n\nDon't ask the model to prove it's correct.\n\nGive it something independent to answer to.", "url": "https://wpnews.pro/news/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying", "canonical_source": "https://dev.to/statementdecoder/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying-to-me-24ik", "published_at": "2026-07-23 16:45:32+00:00", "updated_at": "2026-07-23 17:04:24.738052+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "developer-tools"], "entities": ["StatementDecoder", "Wise", "Revolut", "OFX"], "alternates": {"html": "https://wpnews.pro/news/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying", "markdown": "https://wpnews.pro/news/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying.md", "text": "https://wpnews.pro/news/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying.txt", "jsonld": "https://wpnews.pro/news/i-built-an-ai-first-bank-statement-converter-the-one-rule-that-stopped-it-lying.jsonld"}}