Your AI assistant just generated tests until the coverage bar turned green. 92%. Ship it?
Here's the trap: coverage measures which lines ran, not whether a test would notice when they're wrong. AI made that gap free — you can generate 90% coverage in a minute, all of it happy-path, none of it discriminating. That's coverage theatre: it looks like safety, it measures activity, it proves almost nothing.
A line is "covered" the moment a test executes it. But executing a line and asserting the right thing about it are different claims. An AI-generated test that calls applyDiscount(order) and asserts typeof result === 'number' covers the function — and would stay green if the discount math were completely wrong.
Coverage answers "did this code run in a test?" The question that earns trust is "would a test fail if this code were wrong?" Those are not the same, and AI output routinely nails the first while skipping the second.
If a suite can't tell correct behavior from a real defect, its coverage number is decoration. The cheapest check is mutation: change > to >=, flip a boolean, swap two lines — then see if anything goes red. If the suite stays green on broken code, it isn't protecting you.
I put the smallest runnable version of this in a free lab (no install, Node 20+):
node run-lab.js all
It hands the AI's own suite a pricing function with four seeded defects, then runs an independent verification suite against the same broken code:
WEAK TESTS (the AI's suite) .... PASS 3/3
STRONG VERIFICATION ........... FAIL 8/12 <- catches all 4 defects
MUTATION GATE ................. PASS 4/4
LAB GATE: PASS
The AI's tests pass against code that is wrong. Verification is what fails — and that failure is the signal coverage never gave you.
Coverage isn't useless — low coverage is a real red flag. But high coverage is not evidence of anything on its own, and AI made it trivially easy to manufacture. Treat the green bar as a starting question, not an answer.
Free, self-contained lab: https://github.com/chernevnikolay86-wq/ai-test-verification-lab
I turned the full method into a book + runnable kit — the operating loop, governance, and labs across unit, web/API, instrument-cluster and CAN/UDS domains.
Honest caveat: the lab systems are simulators with declared defects — they prove the method catches faults, not that any product is certified. That's the point: don't trust things that merely look right.