# Coverage theatre: your AI hit 90% coverage and still shipped the bug

> Source: <https://dev.to/nikolay_chernev_a67479931/coverage-theatre-your-ai-hit-90-coverage-and-still-shipped-the-bug-5bml>
> Published: 2026-09-30 09:07:33+00:00

Your AI assistant just generated tests until the coverage bar turned green. 92%. Ship it?

Here's the trap: coverage measures which lines *ran*, not whether a test would notice when they're *wrong*. AI made that gap free — you can generate 90% coverage in a minute, all of it happy-path, none of it discriminating. That's coverage theatre: it looks like safety, it measures activity, it proves almost nothing.

A line is "covered" the moment a test executes it. But executing a line and asserting the *right thing* about it are different claims. An AI-generated test that calls `applyDiscount(order)` and asserts `typeof result === 'number'` covers the function — and would stay green if the discount math were completely wrong.

Coverage answers "did this code run in a test?" The question that earns trust is "would a test fail if this code were wrong?" Those are not the same, and AI output routinely nails the first while skipping the second.

If a suite can't tell correct behavior from a real defect, its coverage number is decoration. The cheapest check is mutation: change `>` to `>=`, flip a boolean, swap two lines — then see if anything goes red. If the suite stays green on broken code, it isn't protecting you.

I put the smallest runnable version of this in a free lab (no install, Node 20+):

```
node run-lab.js all
```

It hands the AI's own suite a pricing function with four seeded defects, then runs an independent verification suite against the same broken code:

```
WEAK TESTS (the AI's suite) .... PASS 3/3
STRONG VERIFICATION ........... FAIL 8/12   <- catches all 4 defects
MUTATION GATE ................. PASS 4/4
LAB GATE: PASS
```

The AI's tests pass against code that is wrong. Verification is what fails — and that failure is the signal coverage never gave you.

Coverage isn't useless — low coverage is a real red flag. But high coverage is not evidence of anything on its own, and AI made it trivially easy to manufacture. Treat the green bar as a starting question, not an answer.

Free, self-contained lab: [https://github.com/chernevnikolay86-wq/ai-test-verification-lab](https://github.com/chernevnikolay86-wq/ai-test-verification-lab)

I turned the full method into a book + runnable kit — the operating loop, governance, and labs across unit, web/API, instrument-cluster and CAN/UDS domains.

Honest caveat: the lab systems are simulators with *declared* defects — they prove the method catches faults, not that any product is certified. That's the point: don't trust things that merely look right.
