I handed an AI agent 13 invoices and a list of what we had actually ordered. It approved 10 of them, $9,049.05, and refused 3. The 3 it refused were the exact 3 I had rigged to be wrong: a padded total, an invoice nobody ordered, and a straight duplicate. Together they came to $1,411.25 in payments that should never have gone out.
The satisfying part is not that it caught them. It is that the AI did almost none of the deciding.
13 invoice images across 4 different layouts, including one deliberately awful scan (crooked, low contrast, the kind of thing a phone camera produces at a bad angle on a bad day). A vision model reads each image into strict JSON. Then plain code, no AI anywhere in it, checks that JSON against the purchase orders we placed.
The stack:
The 13 reads took 28 seconds end to end. At roughly a cent an image, the model cost here is a rounding error. The value is entirely in what happens after the read.
The gate checks every invoice against three conditions, and all three have to pass:
Here are the three problems I planted, and which rule stopped each one:
Notice what none of these required. None of them needed intelligence, judgment, or a model. They needed a lookup and a comparison. That is the whole point.
The awful scan bit me, and I kept it in the numbers because pretending otherwise would defeat the purpose of this channel.
The invoice date printed as 02/08/2026
. The model read it as February 8. It was meant to be August 2. Classic US-versus-EU date order, and the model picked wrong with complete confidence. It did not flag any uncertainty. It handed me a clean, plausible, internally consistent February 8 and moved on.
Nothing broke, because my gate does not use dates for anything. But that is luck, not design. If you build anything that touches due dates, payment terms, or late fees, the date field is the one you hand-verify, because a wrong date will sail through every arithmetic check you have.
The broader lesson is the one worth carrying: a vision model will give you a wrong value with the same confidence it gives you a right one, and the wrong value will often be internally consistent. Confidence is not correctness, and there is no exception in the JSON telling you which fields to trust.
Vision extraction is probabilistic. Every read is a best guess, and best guesses are wrong some fraction of the time in ways you cannot predict per-invoice. So the rule I follow is simple:
Never let a probabilistic component make a deterministic decision about money.
Keep the model's job narrow. Its only responsibility is turning pixels into structured data. Every decision with a consequence lives in code you can read, test, and reason about. The model votes. The code decides. When those two things blur together, you have built something that pays $118.80 it should not have, and you will not find out until you reconcile the statement.
A few things that made this work in practice:
You do not need a smarter model to build this. You need a dumber, stricter layer sitting between the model and the bank.
The full n8n workflow, the code node, and the 13 test invoices (including the awful scan) are here: github.com/Ships-Itself/builds/tree/main/ep04-invoice-agent.
If you want to watch this one get built and stress-tested end to end, the video lives on youtube.com/@shipsitself.