# The agent that refuses to guess > Source: > Published: 2026-08-31 03:24:41+00:00 I work at a B2B telecom consultancy. I'm not the one auditing the bills, but every month I watch how it's done: open the invoice PDF, check every line against the signed contract, compare it with what the account used in earlier cycles, and write up whatever doesn't add up. It's slow, it doesn't scale, and it's the first task dropped when the month gets busy — which is exactly when the money leaks. And outside a consultancy it's worse: most companies just pay the bill because it arrived. So I built an agent that does the whole job. One invoice PDF in; a dispute letter for the carrier and an executive summary for the customer out, with nobody in the loop. It runs on Gemini 3.5 Flash and the Google ADK, on Cloud Run and Firestore, and it's open source: [github.com/Bren0-lz/invoice-sentinel](https://github.com/Bren0-lz/invoice-sentinel). This post is about the one decision that shaped everything else, and about the four defects that only showed up when I stopped reading my own code and started attacking the running service. An agent that writes a dispute letter is writing a document addressed to a third party and signed by the customer. If a figure in it is invented, the customer doesn't lose a feature — they lose credibility with their own supplier, and they lose it in writing. So the rule is absolute, and it isn't a line in a prompt. It's enforced in three layers, each of which would have to fail independently: **Structurally.** The rule engine is pure Python with `Decimal` . No module under `rules/` imports an LLM client. Five rules across three families, running concurrently under a `ParallelAgent` . **In the tool signatures.** No auditor tool accepts a monetary value as an argument. `flag_anomaly(finding_id, rationale)` cannot be talked into disputing four thousand reais that nobody computed, because there is no parameter to put it in. A test asserts this with `inspect.signature` , so the guarantee survives someone adding a tool later. **In the generated prose.** `amount_guard` extracts every money-shaped number from the letter and checks it against the set the engine actually computed. Invented one? The model is told *which* number and rewrites it. Insisted past the rewrite budget? The dispute is stored as `blocked` — never as a `draft` somebody might later mistake for reviewed. The third layer is the one people forget, and the one that matters most. The first two make the model unable to author a figure; only the third checks what it actually wrote. The agent ships layout profiles for two carriers. Send it a bill from a third and it refuses, rather than reading it with someone else's separator hints — because a Brazilian `1.234,56` read as American is `1.234` , and that is the shape of error nothing downstream catches. It's plausible. It's wrong. It's a thousand times too small. The same principle runs through the whole system: That last one was a real defect, and it's the best lesson in the project. Every one of these came from using the deployed service the way an evaluator would, with documents nobody prepared knowing what would be checked. With no contract on file, three of five rules are skipped and the findings list comes back empty. The prompt then told the model to say the invoice "looks clean". A brand-new account is the first path anyone walks. It got that certificate over a bill that had a real overcharge — which appeared the moment the contract was filed. Worse, the rule for escalating a missing contract existed ten lines below in the same prompt and was **unreachable**: escalation needs a `finding_id` , and there are no findings. **Silence from a rule that never ran is not evidence that the bill is correct.** Empty now branches on whether a contract exists, before it can mean anything. Transcribing a three-plan contract, it filed six — helpfully re-adding each plan under the abbreviated name the invoice prints, so whoever had to match the two documents would have an easier time. The contract is the baseline every figure is computed against. It now contained terms nobody signed. Fixed with a validator that rejects duplicate plans and a repair loop that sends the model back, not with a sterner prompt. A Brazilian invoice was transcribed perfectly and still reported 149.42 out of balance. ICMS, PIS, COFINS, FUST and FUNTTEL are computed *inside* the price here and itemised only because Lei 12.741/2012 requires it. Summing them double-counts. The schema now declares the tax regime per carrier — inclusive in Brazil, additive in the US — so the balance warning means what it says instead of firing on every bill from a country whose invoices are all built that way. This is my favourite, because I found it by going looking. The parser that coerces a transcribed amount into `Decimal` strips currency noise with a regex that keeps only digits, separators and an **ASCII hyphen**. So: | Printed on the bill | Parsed as | Should be | |---|---|---| `−50,00` (U+2212, what a properly typeset PDF prints) | `+50.00` | `-50.00` | `–50,00` (en dash, from a layout pasted out of a word processor) | `+50.00` | `-50.00` | `—50,00` (em dash) | `+50.00` | `-50.00` | `(50,00)` (accounting notation) | `+50.00` | `-50.00` | A fifty-real credit became a fifty-real charge, and the invoice total it reconciles against moved by a hundred. Here's what makes it worth writing about: **nothing downstream could have caught it.** The three layers protecting the money protect against the *model* authoring a figure. They say nothing about a figure entering wrong. `amount_guard` compares the letter against what the engine computed — it cannot compare the engine against the page. The sign was gone before the value was ever a `Decimal` . The parser had no tests at all. The deterministic safety net under the project's central claim, with zero coverage. It now normalises the dash forms and reads parentheses as negative. And a value that states its sign twice — `(-50,00)` — is **refused** rather than resolved, because picking one would be guessing about money. **Make the guarantee structural, not textual.** Every time I needed something to be true, a prompt was the weakest place to put it. A validator, a function signature or a type has to be actively defeated; a prompt just has to be forgotten. **Test the deployed thing, with documents you didn't make.** My whole synthetic dataset comes from one generator, so it proves the rule engine agrees with the generator — a weaker claim than it looks. The first time I audited an invoice built independently, three defects surfaced at once, and all three were *false positives*: findings that weren't there. That document is now committed as a regression fixture, as the false-positive control with the generator taken out of the loop. **Watch what a refusal actually stops.** I added a refusal at the intake stage and it changed nothing: sibling stages of a `SequentialAgent` can't be skipped from outside, and the extractor was reading the attachment on its own and extracting anyway, right past the "no". A refusal is a text in the transcript until something downstream asks. **Count your model calls, honestly.** Only three of eleven pipeline stages are `LlmAgent` s. Deciding whether an attachment is a contract or a bill is pattern matching; so is picking a carrier profile off a letterhead. A model call whose only job is to invoke a function is tokens spent on nothing, and it dresses deterministic work up as reasoning. Measured on 15 synthetic invoices, 4 accounts, 2 visually distinct carrier layouts: Clone it and run `pytest` ; the accuracy claims verify themselves against extractions committed in the repo. Honest limitations, since I'd rather say them than have you find them: two carriers only, the dataset is synthetic, and the audited PDF isn't stored yet, so an audit can't be re-checked against the document that produced it. Built for the [All Things Agentic Hackathon](https://allthingsagentichackathon.devpost.com/), track