Back when I first delegated QA to an AI agent, it signed off on a tool with "all features working, pass." I opened the tool myself. The canvas was blank.
The AI wasn't lying. In the environment it was looking at, the tool genuinely appeared to work.
I run visual QA across a large fleet of web tools using Claude + Chrome MCP, and this class of false positive traced back to exactly two causes.
Chrome MCP typically operates in hidden (background) tabs. And browsers throttle requestAnimationFrame
aggressively in hidden tabs to save power.
QA a canvas or animation feature in that environment and here's what happens: the JS executes, no errors fire, event handlers respond — and the render finishes at zero frames. The AI looks at a screenshot plus clean JS results and concludes "animation started, no errors, pass." Code executing and pixels rendering are different events, and a hidden tab erases the distinction.
I didn't want this to be a "back in my day" post, so I re-ran the measurement right before publishing. Opened a tab via Chrome MCP, ran an rAF counter:
| Measurement | Result |
|---|---|
| document.visibilityState | hidden |
| rAF fires in 3.37s | 0 |
| setInterval(100ms) fires, same window | 4 (expected: 33) |
| setTimeout(3000ms) actual delay | 3373ms |
rAF wasn't throttled. It was stopped — zero frames.
And as a bonus finding: setInterval
was decimated to roughly 1/8 of its expected rate, and even setTimeout
drifted. So it's not just rAF-based animation; timer-driven logic in general cannot be trusted in a hidden tab.
active: true
so they're visibleThe second trap shows up in AI QA reports as pass rationale like this:
onclick wiring confirmed. Zero JS errors. Library loaded. → PASS
All of that describes code-path health, not feature behavior. Wiring can be correct while nothing renders (Cause 1). Errors can be absent while the downloaded file is empty.
My fix: grep the template for dynamic features first, then require a behavior check for every hit.
| Found in code | Required behavior check |
|---|---|
<canvas> |
|
| Interact in a visible tab → screenshot after a delay shows rendered content | |
<input type="file"> |
|
| Upload a dummy file → preview src is non-empty | |
download attr / toBlob |
|
| Press the button → observe Blob creation | |
mousedown / touchstart |
|
| Fire the event → state/transform actually changed | |
requestAnimationFrame |
|
| Transform values differ after time passes |
QA reports must include a feature-coverage table, and "exists in code but behavior unverified" can never be graded as pass. That's the whole rule.
That alone cut the false positives dramatically.
A side benefit of visual QA: cross-checking what the code claims against what the pixels show. "The class is text-secondary
(cyan) but it renders gray" — cascade and override bugs get caught with evidence attached. Humans tend to stall at "the color feels off somehow."
Hidden-tab throttling is a browser power-saving behavior, so Chrome could change it. My operating principle isn't "hidden tabs are bad" — it's "dynamic rendering must be verified in a visible state."
Behavior checks cost time and tokens. Running them on every feature of every tool is too heavy; I only require them for tools where the grep finds dynamic features.
Verified: April–June 2026 in production, re-reproduced July 10, 2026. Environment: Claude + Chrome MCP / Windows 11.