# QA Agents That Test the App Themselves: More Bugs Found, at $2.67 per Bug > Source: > Published: 2026-11-03 02:12:20+00:00 # QA Agents That Test the App Themselves: More Bugs Found, at $2.67 per Bug The short answer: letting agents test the app themselves finds more real bugs than writing test scripts — but each bug costs more, and precision is slightly lower. The post introduces Momentic's Mo, a custom agent harness built for catching bugs: as it explores, it builds a self-organizing directed acyclic graph, and large product areas automatically fork into parallel sub-agents, each with its own browser or device for concurrent execution. Planner agents work from screenshots and video, while locator agents inspect page structure to turn plain-English targets into actions. When several agents run at once, they share resources under a lease system: shared use is allowed, but only one agent may change a given resource at a time. For an ai software engineer considering agents for QA, this architecture is the starting point for understanding both the capability and the cost. The benchmark gives the comparison something to stand on. Four popular open-source apps — Gitea, Metabase, Twenty, and Saleor — were tested against 1,396 human-labeled bugs, with each agent given the same brief, a fresh sandbox, and no access to source code, in a time-boxed 60-minute run. Mo found 269 bugs, with 19.3% recall, 84% precision, and $2.67 per bug. Codex found 86 bugs, with 6.2% recall, 86% precision, and $0.80 per bug. [Claude Code](https://promptcube3.com/en/tags/claude%20code/) found 59 bugs, with 4.2% recall, 80% precision, and $4.09 per bug. Devin found 27 bugs, with 1.9% recall, 84% precision, and $3.89 per bug. Mo's edge is recall: it surfaces the most real bugs. Its costs are precision that is not the best in the group and a per-bug price above Codex. Time changes the picture further. From 30 minutes to 60 minutes, Mo went from 177 bugs to 269 bugs, a gain of 52%, while Codex gained only 16%. More time helps Mo more than it helps the others, but time also accumulates cost. Two limits belong in any honest reading. The evaluation used a language model as judge with a human reviewer, and the result comes from one time-boxed run on four apps, so it does not prove the same outcome on every product. The post also notes that language models generating code tend to reinforce their own assumptions, write many useless tests, and struggle with visual understanding — the very problems agent testing aims to fix, and a reason it can still go wrong. For an ai software engineer deciding whether to adopt this approach, the practical move is a small pilot on your own app: measure recall, precision, and cost per bug, then choose full adoption, assisted scanning, or staying with scripted tests. [Next What Changed in Tech Job Ads After ChatGPT: Counting Tools in 118,545 Indonesian Ads →](https://promptcube3.com/en/threads/10213/) ## All Replies （0） Want a live back-and-forth? [Join the global AI chat room](https://promptcube3.com/en/chat/) — login to talk. No replies yet — be the first!