{"slug": "my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was", "title": "My AI QA agent said \"all features working.\" The canvas was blank. Here's what it was actually seeing.", "summary": "An engineer discovered that AI-driven visual QA agents can produce false positives when testing canvas and animation features in hidden browser tabs. Chrome aggressively throttles requestAnimationFrame and timer-driven logic in background tabs, causing code to execute without rendering any pixels. The developer implemented behavior checks for dynamic features, requiring verification in visible tabs, which dramatically reduced false positives.", "body_md": "Back when I first delegated QA to an AI agent, it signed off on a tool with \"all features working, pass.\" I opened the tool myself. The canvas was blank.\n\nThe AI wasn't lying. In the environment it was looking at, the tool genuinely appeared to work.\n\nI run visual QA across a large fleet of web tools using Claude + Chrome MCP, and this class of false positive traced back to exactly two causes.\n\nChrome MCP typically operates in hidden (background) tabs. And browsers throttle `requestAnimationFrame`\n\naggressively in hidden tabs to save power.\n\nQA a canvas or animation feature in that environment and here's what happens: the JS executes, no errors fire, event handlers respond — and the render finishes at zero frames. The AI looks at a screenshot plus clean JS results and concludes \"animation started, no errors, pass.\" Code executing and pixels rendering are different events, and a hidden tab erases the distinction.\n\nI didn't want this to be a \"back in my day\" post, so I re-ran the measurement right before publishing. Opened a tab via Chrome MCP, ran an rAF counter:\n\n| Measurement | Result |\n|---|---|\n| document.visibilityState | hidden |\n| rAF fires in 3.37s | 0 |\n| setInterval(100ms) fires, same window | 4 (expected: 33) |\n| setTimeout(3000ms) actual delay | 3373ms |\n\nrAF wasn't throttled. It was stopped — zero frames.\n\nAnd as a bonus finding: `setInterval`\n\nwas decimated to roughly 1/8 of its expected rate, and even `setTimeout`\n\ndrifted. So it's not just rAF-based animation; timer-driven logic in general cannot be trusted in a hidden tab.\n\n`active: true`\n\nso they're visibleThe second trap shows up in AI QA reports as pass rationale like this:\n\nonclick wiring confirmed. Zero JS errors. Library loaded. → PASS\n\nAll of that describes code-path health, not feature behavior. Wiring can be correct while nothing renders (Cause 1). Errors can be absent while the downloaded file is empty.\n\nMy fix: grep the template for dynamic features first, then require a behavior check for every hit.\n\n| Found in code | Required behavior check |\n|---|---|\n`<canvas>` |\nInteract in a visible tab → screenshot after a delay shows rendered content |\n`<input type=\"file\">` |\nUpload a dummy file → preview src is non-empty |\n`download` attr / `toBlob`\n|\nPress the button → observe Blob creation |\n`mousedown` / `touchstart`\n|\nFire the event → state/transform actually changed |\n`requestAnimationFrame` |\nTransform values differ after time passes |\n\nQA reports must include a feature-coverage table, and \"exists in code but behavior unverified\" can never be graded as pass. That's the whole rule.\n\nThat alone cut the false positives dramatically.\n\nA side benefit of visual QA: cross-checking what the code claims against what the pixels show. \"The class is `text-secondary`\n\n(cyan) but it renders gray\" — cascade and override bugs get caught with evidence attached. Humans tend to stall at \"the color feels off somehow.\"\n\nHidden-tab throttling is a browser power-saving behavior, so Chrome could change it. My operating principle isn't \"hidden tabs are bad\" — it's \"dynamic rendering must be verified in a visible state.\"\n\nBehavior checks cost time and tokens. Running them on every feature of every tool is too heavy; I only require them for tools where the grep finds dynamic features.\n\n*Verified: April–June 2026 in production, re-reproduced July 10, 2026. Environment: Claude + Chrome MCP / Windows 11.*", "url": "https://wpnews.pro/news/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was", "canonical_source": "https://dev.to/mxhlix/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-heres-what-it-was-actually-1pl5", "published_at": "2026-07-17 02:12:49+00:00", "updated_at": "2026-07-17 02:27:50.593204+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "computer-vision"], "entities": ["Claude", "Chrome MCP", "Windows 11"], "alternates": {"html": "https://wpnews.pro/news/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was", "markdown": "https://wpnews.pro/news/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was.md", "text": "https://wpnews.pro/news/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was.txt", "jsonld": "https://wpnews.pro/news/my-ai-qa-agent-said-all-features-working-the-canvas-was-blank-here-s-what-it-was.jsonld"}}