An AI Assistant Handed Me 6 'Real' Invoice Samples. 5 Were Completely Made Up

A developer building a parser for Vietnamese electronic invoices asked an AI assistant to gather six sample invoice XML files from well-known providers, and the parser rejected five of them. Investigation revealed the five rejected files were AI-fabricated test fixtures using invented English tag names like <SellerInfo> and <BuyerTaxCode>, while only the sixth used the government-mandated Vietnamese tag structure (TTChung, NBan, NMua, MST, DChi) that all providers are legally required to use since 2022. Testing against real invoices later surfaced three bugs, including a privacy leak where a raw fallback dump bypassed redaction rules for individual buyers without tax codes.

I was building a parser for Vietnamese electronic invoices — the XML files tax authorities require for every VAT invoice issued in Vietnam. The goal: read a file from any provider, whoever issued it, and normalize it into one clean JSON shape. To sanity-check it against real-world variation, I asked another AI assistant to help me gather sample invoice XML files from a handful of different well-known Vietnamese e-invoice providers, plus one generic "tax authority standard" file — six in total. I ran my parser against all six. It rejected five of them outright. My first reaction was: great, my code is broken on day one, five different providers, five different failures. My second reaction, a few minutes later, was that this was actually the correct outcome — and figuring out why took me somewhere I wasn't expecting. Before assuming my code was wrong, I did something boring: I read the README the other AI had written for the sample files. Buried in it: "the data inside company names, tax codes, amounts is simulated test data, not actual issued invoices... sourced from an open-source repo." Simulated. Not "close to real." Simulated by a developer who'd never seen an actual Vietnamese invoice, writing plausible-looking test fixtures for their own unrelated project on GitHub — repurposed by an AI assistant as "sample invoices" because they were shaped like the right kind of thing. So instead of trusting my own tests, I went and checked the actual tag names against the legal source of truth. Here's the thing about Vietnamese e-invoices: since 2022, every provider is legally required to use the exact same XML tag structure, defined by the tax authority. Not "similar." Identical. TTChung , NBan , NMua , MST , DChi — Vietnamese abbreviations, mandated by decree, because the tax authority's own systems have to parse every invoice regardless of which vendor issued it. Five of my six "sample" files used tags like <SellerInfo , <BuyerTaxCode , <InvoiceNumber . Clean, readable, very sensible-looking English tag names. Tag names that, as far as I can tell, no real Vietnamese e-invoice has ever used. Nobody in Vietnam is legally allowed to invent their own — that's the entire point of the mandate. My parser hadn't failed. It had correctly identified five pieces of confident, internally-consistent fiction and said so. Only the sixth file used the real government-mandated structure, and that's the only one it accepted. I could have made the parser "smarter" — taught it to also recognize the invented English tag names, so it would accept all six and I'd have a tidier-looking demo. I didn't. Teaching a parser to accept a format that doesn't exist in reality isn't robustness. It's just a more elaborate way of being wrong, with extra confidence — and the first real invoice that came in later would have had to fight through that noise instead of matching cleanly. So I built strictly to the real legal spec, kept the one sample that actually matched it, and waited for real data. Over the next few days, the person I was building this for started digging up actual XML files from their own email and provider portals — a utility bill, a couple of online purchase receipts, a clinic invoice, an old 2021 logistics invoice, eventually a genuine business-to-business invoice. Three bugs surfaced. None of them showed up on a single one of the AI-generated samples, because you can only find these by touching the real thing. Bug 1 — the privacy leak. My parser had a "raw fallback" mode: alongside the clean normalized output, it also dumped every tag/value pair it found, so nobody would lose data to a field I hadn't mapped yet. I'd also built a privacy feature — if a buyer had no tax code a strong signal they're a private individual, not a business , their name and address got redacted from the output. Redacted from the clean output, that is. Nobody told the raw fallback dump about that rule. The first real invoice I tested — a utility bill — leaked the buyer's actual name, home address, and email straight through the "debug" field I'd added for convenience. The privacy feature and the debug feature had never been told about each other. Bug 2 — silently dropping the buyer's name. The real spec has two different tags for "who bought this": one for a company name, a different one for an individual's name. I only knew about the company-name tag — none of the fake samples had ever used the other one, because whoever wrote them didn't know it existed either. Real invoices from individual buyers started coming in using that second tag. My parser read them, found nothing under the tag it was looking for, and happily reported "fully processed, no issues." It wasn't lying exactly. It just didn't know what it didn't know. Bug 3 — the one that actually worried me. A 2021 invoice came in using an older schema version than everything I'd tested. Same overall shape, but the invoice number and issue date — the two fields you'd use to identify or deduplicate an invoice — lived under different tag names entirely. My parser again reported total success while silently returning null for both. That's the one that made me change how the whole system reports errors. It's not enough to check "did I find the big structural block." I had to start checking "did I actually get a value for the fields that matter," and say so explicitly when I didn't — instead of a green checkmark hiding a null. The fake samples weren't malicious, and the AI that made them was upfront that they weren't real — I just could have easily not checked. They were also useless as an adversary . A genuinely broken test file breaks loudly. A plausible-but-fictional one passes quietly and teaches you nothing, because it was generated by something optimizing for "looks like a Vietnamese invoice" instead of "is one." Real data doesn't have that problem. It doesn't care if it's plausible. It just is what it is, including two tag names I'd never heard of and a schema version I didn't know still existed in the wild. I don't think the lesson here is "don't use AI-generated test data." I used AI for plenty of this build, including help gathering that first batch of samples. The lesson is narrower: when you're validating against an external, legally-defined standard you don't fully control, "generated a file that passes my own tests" and "matches reality" are two different claims, and only one of them is checkable — by going back to the source of truth instead of trusting the fixture. Wait, so is the AI-generated test data "bad"? Not really — it was honestly labeled as simulated, and it was still useful as a starting shape. The mistake would've been mine, not the tool's, if I'd stopped checking there. What does the parser do now when it hits a tag name it's never seen? It doesn't guess. It reports exactly what it found, flags which expected fields came back empty, and includes the raw data it read so a human can see what actually happened instead of a silent null. Couldn't you just support every tag name variant you find? For invented, non-standard ones, no — that would mean trusting fiction as if it were law. For real variants tied to an older but still legally valid schema version, yes, and that's exactly what happened with bug 3. Is this a Vietnam-specific problem? The specific tags are. The pattern — an AI-shaped test fixture that's plausible enough to pass your own tests without matching the actual external standard you're building against — isn't. The parser is live on Apify Store as Vietnam E-Invoice XML Normalizer https://apify.com/donerightlabs/vn-einvoice-xml-normalizer — callable directly by AI agents over MCP if you're building bookkeeping tools that need to ingest Vietnamese invoices.