It Passed Every Test. That's Why It Can't Ship Yet.

An engineer's order-reading LLM passed a 29-question exam with zero fatal errors, but the developer refused to ship it immediately because the exam was authored by the same person, covering only imagined scenarios. The developer implemented a guarded deployment where human review catches all results initially, and every miss becomes a new test case, emphasizing that production data reveals the real exam. The process also uncovered five mistakes in the exam itself, showing that the test harness caught more author errors than model errors.

My order-reading LLM passed the 29-question exam. Zero fatal errors. The model is chosen https://dev.to/ramses203/the-model-that-costs-3x-more-won-by-exactly-one-question-55aj . So — ship it? No. And the reason is the most important thing in this series. The questions, the answer key, the product catalog — I made all of it. Which means "passed 29 questions" translates precisely to: "It made no mistakes in the 29 situations I was able to imagine." Nothing more. The sentence I couldn't imagine is not on the exam. And production is a parade of sentences you couldn't imagine. Real customers abbreviate in ways I can't invent, and real product catalogs are far dirtier than the one I wrote — the same product registered three times under different names, dead items nobody deleted piling up for years. Before shipping, I wrote these three lines down. VERIFIED one question per worst-accident type; both models at zero NOT VERIFIED never ran on production data. Every exam sentence is mine GUARDED when unsure, it must not confirm — it hands off to a human The third line is the one that matters. If the unknown case falls somewhere safe by construction, an incomplete exam is still shippable. When a never-seen phrasing arrives, this program's worst case is "slower" — not "wrong goods shipped." Without that guard, shipping on exam scores alone is trusting a gun because it fired 29 times without jamming. For a while after launch, nothing passes through automatically. A human reviews every result. What comes out of this period is the real exam. The sentences I couldn't imagine make their first appearance here. Every miss becomes a new exam question. This is the moment the exam grows from imagination into production. Week 1 humans check everything. Every miss becomes a test case Weeks 2–4 if FATAL holds at 0, auto-pass the confirmed ones only After humans only see the "needs confirmation" queue There's an opposite case: FATAL keeps appearing, and the cause isn't the prompt — it's the question itself. "Figure out what 'the usual' means, automatically" is that kind of question. It cannot be done in principle. The information isn't there. No amount of prompt polish fixes it, and every polish makes it worse by teaching the model to guess with confidence. The move there isn't fixing the program. It's shrinking the scope. Draw the line — "that case goes to a human" — and automate the rest. Finding out what is impossible in principle is also the exam's job. Counting what this exam actually caught: The model's real mistakes — 1. And its only crime was asking. The exam author's mistakes — 5. Three in the answer key, two in the grader. The exam I built to verify the AI caught me more than it caught the AI. Which is the real reason to build one. Passing is the starting line. Production writes the next questions. P.S. The scoreboard above changed again later, when the grader got fixed one more time. That story is its own post — it's here. All 29 questions and the grader are public → github.com/ramses203/llm-test-harness