# Counting bugs is the hard part of comparing AI review tools > Source: > Published: 2026-09-16 00:15:10+00:00 Model benchmarks for code review usually report the same two numbers: how many bugs the tool found, and how many of its comments were actually right. The part nobody can verify is the counting. Who decided a finding was a real bug? That single choice moves precision by more than the model choice does. The Entelligence benchmark comparing GPT-5.6 Luna and GPT-6 Astra on review is a useful case, because it is one of the few vendors that publishes its method. Ran on 2026-09-14, it put both models against 50 public PRs from the AI-Code-Review-Evals dataset (Cal.com, Sentry, Discourse, Keycloak, Grafana). Across those PRs Luna reported 69 verified bugs and Astra 92. Luna's precision was 74%, Astra's 96%. Total review cost was $0.20 for Luna and $5.66 for Astra. The headline is that a model roughly 28 times cheaper misses fewer than a third more bugs. The part worth staring at is the verification. Entelligence had two judges score every finding, and the two agreed on 91% of them. A bug only counted when both judges called it real. That is the right shape of a method, and it is rare in the category. Most comparison posts that reach you quote a recall figure with no statement of who labeled the ground truth, whether the labeler worked for the vendor, or how two judges resolved disagreement. Here is why the counting matters more than the model. A correctness label has a huge surface: a finding can be a true bug, a style nit, a misunderstanding of the code, or a duplicate of something a human already flagged. Where a vendor draws those lines determines precision. The same model run could get 80% or 95% precision depending on whether the checker counts near-duplicates as hits. So "96% precision" without the label protocol is not a number you can act on. It is a claim[reference:[https://entelligence.ai/blogs/gpt-5.6-luna-vs-gpt-6-astra-is-a-1.20-model-good-enough-for-code-review](https://entelligence.ai/blogs/gpt-5.6-luna-vs-gpt-6-astra-is-a-1.20-model-good-enough-for-code-review)]. The practical consequence for a team: when you read a tool comparison, ask for three things before trusting a precision or recall figure. The label protocol out loud, not a link buried in an appendix. Whether the judge who built the product also scored it, since a vendor judging its own output is a conflict, even an honest one. And the disagreement rate between judges, because 91% agreement means 9% of the hard calls went the scoring team's way, and those are exactly the security-critical findings you care about. None of this tells you which tool to buy, and it does not tell Entelligence's models what to fix when precision drops. On Keycloak's auth code, Luna's precision fell toward 50%, meaning a working review on permission logic was flipping a coin on whether a comment was real[reference:[https://agentwrotethis.dev/blog/multi-repo-ai-review-a-context-problem-not-a-volume-problem/](https://agentwrotethis.dev/blog/multi-repo-ai-review-a-context-problem-not-a-volume-problem/)]. The counter for that is not a better benchmark. It is a verify step a human owns. The takeaway is operational: treat a review-tool recall figure as unverified until you know who called the ground truth. Vendor benchmarks in this category publish method so rarely that the ones that do are worth reading twice, and the ones that do not are worth ignoring until they do.