# My 9-Reviewer AI Gate Failed Articles It Scored 9.1 > Source: > Published: 2026-08-26 00:01:20+00:00 Nine LLM reviewers score every article I publish. The gate is simple: zero blockers, and a mean score above my floor. Yesterday it failed an article it had scored **9.1**. Then it failed a second one three separate times. Neither reviewer ever disagreed with the writing — all three failures were the harness misreading its own reviewers. The Checklist reviewer returned **10.0** and wrote this under `BLOCKERS` : ``` no biography/family/military/location/heritage; no political position; the numbers are incident/operational facts, so the rule doesn't bite. ``` That is a clean bill of health. It failed the gate. The parser dropped lines meaning "nothing here" with `/^none/i` . The prompt asks reviewers to *explain* why nothing is wrong, so they write the explanation instead of the word — and **"no biography" is not "none"**. **Why it survives review:** the regex is correct for the string it was written against. Nobody writes a test for the sentence a model *didn't* emit. The fix wasn't a longer regex. Guessing every phrasing of "nothing is wrong" is unbounded. The prompts already state the contract — fully in scope means `BLOCKERS: None` **and** `SCORE: 10` — so a 10 that also reports a blocker is self-contradictory: ``` if (score === 10 && blockers.length) { improvements.push(...blockers); // demote, never drop blockers = []; } ``` Demoted, not deleted. If a reviewer ever scores 10 on something genuinely broken, the text still reaches a human. My runner treats a quota message as a transport failure, so a billing error can never be scored as a bad article. The pattern included a bare `rate limit` . It was tested against the reviewer's **entire output**. My corpus is security articles. Reviewers say "rate limit" constantly: ``` · Discovery & Hook: USAGE LIMIT — SCORE: 8.0 · Voice & Agenda: USAGE LIMIT — SCORE: 9.5 · Quality: USAGE LIMIT — SCORE: 9.0 ``` Three finished reviews, scores visible in the text that got thrown away. That article lost 4 of 9 reviewers and failed two batch runs before I looked. It reads as an account problem, which is exactly why it survived. The tell separates the two cleanly: a real quota banner is the *only* thing on stdout. A review that merely discusses limits has a `SCORE:` line sitting right there. One reviewer is informational — its rubric weights brand fit 40%, which structurally caps anything outside that niche. It was excluded from the gate *score*. It was not excluded from blockers. So it vetoed through the back door — and what it files under `BLOCKERS` is its own arithmetic: ``` ✗ Axis 1 — Content Quality: 8.5 ✗ Weighted = 8.5 x 0.6 + 5 x 0.4 = 7.1 ``` Four "blockers" on an article it had just called original and reproducible. A half-applied exclusion is not an exclusion — and a [weighted composite](https://ofriperetz.dev/go/composite-scores-and-weighting?utm_source=devto&from=llm-judge-gate-false-negatives) is exactly the shape that hides one, because the arithmetic looks like a finding. Every one of these is the harness misreading agreement as disagreement, and all three fail in the same direction: **quietly, toward rejection.** A false pass is loud — something bad ships and you see it. A false *fail* looks exactly like a strict gate doing its job, so it can run for weeks while you assume the work is merely not good enough yet. It is the same asymmetry that makes [a leaderboard wrong in the flattering direction](https://ofriperetz.dev/go/we-ranked-5-ai-models-by-security-the-leaderboard-is-wrong?utm_source=devto&from=llm-judge-gate-false-negatives), and the same reason [measurement bias](https://ofriperetz.dev/go/bias-in-measurement?utm_source=devto&from=llm-judge-gate-false-negatives) survives longest when it agrees with what you expected. If you run an LLM as a judge, assert on the disagreements it cannot logically have: a perfect score with a blocker, a reviewer excluded from the score that still blocks, a transport error carrying a parsed result. Those are contradictions, and contradictions are testable without predicting a single word the model will say. ``` assert.equal(parse("SCORE: 10\nBLOCKERS:\n- none found here").blockers.length, 0); assert.equal(parse("SCORE: 7\nBLOCKERS:\n- the claim has no date").blockers.length, 1); ``` Both directions. A one-sided test would have passed on all three bugs — my earlier fix for the *opposite* failure is what introduced the second one. That is the same lesson [ground truth taught me that unit tests could not](https://ofriperetz.dev/go/what-ground-truth-caught-that-unit-tests-missed?utm_source=devto&from=llm-judge-gate-false-negatives): a test written from the failure you already know about only ever proves you fixed that one. More on the tooling behind this at [github.com/ofri-peretz/eslint](https://ofriperetz.dev/go/gh/ofri-peretz/eslint?utm_source=devto&from=llm-judge-gate-false-negatives). The three defects above were found on 2026-08-11 across 39 gated articles. *What's the last false negative you found in your own tooling — and how long had it been running?*