Don't Let the AI Find Your Bugs. Let It Judge Them.

A developer building an LLM-based vulnerability scanner explains why he deliberately designed it to flag safe code, arguing that deterministic taint tracking with false positives is more trustworthy than AI agents that skim codebases and hallucinate vulnerabilities. He rejects the common approach of pointing an AI agent at a codebase because LLMs produce inconsistent results, are poor at exhaustive search, and invent findings.

My vulnerability scanner flagged this Java method as SQL injection: String id = request.getParameter "id" ; // user input if id.matches " 0-9 +" { // digits only, or throw throw new IllegalArgumentException "id must be numeric" ; } String sql = "DELETE FROM products WHERE id = " + id; stmt.executeUpdate sql ; // flagged as SQL injection Look at line 2. The input must be pure digits or the method throws. You cannot inject SQL through 0-9 + . There is no attack here. The scanner flagged safe code. Here's the part that sounds wrong: I designed it to do that. The false alarm is not a bug in my scanner. It's the plan. Let me explain, because this decision is the entire foundation of the scanner — and I think it's the decision most people building "AI security tools" right now are getting backwards. I'm building an LLM-based vulnerability scanner — in public, like everything I ship. The pitch is simple: AI writes a lot of code now, fast, often for people who aren't security experts. That code ships with holes. Someone has to find them. So the obvious idea — the one I started with — is the one you've seen in a hundred launch posts this year: "Point an AI agent at your codebase. It reads everything and finds your vulnerabilities." I killed that idea before writing a line of code. Not because it doesn't sound amazing. Because I couldn't defend it with numbers — and the reasons it can't be measured are the same reasons you shouldn't trust it in production. New to this?Two words carry the whole article. Asourceis where untrusted input enters your program request.getParameter . Asinkis where it becomes dangerous executeUpdate — running a database command . A vulnerability is untrusted data reaching a sink without being cleaned on the way. That's it; everything below builds on those two words. Ask an LLM to audit a codebase and you hit three walls. Wall 1: You get a different answer every run. LLMs are probabilistic. Same repo, same prompt, run it twice — the bug lists don't match. Which run goes in the security report? Which one do you benchmark? When a tool's output changes between runs, you can't measure it, you can't compare it to Semgrep or CodeQL, and you can't do science with it. You can only do demos. Wall 2: Searching is the thing LLMs are bad at. Careful, exhaustive enumeration over a large space — visit every file, check every call, miss nothing — is exactly what LLMs don't do. They skim. They fixate. They get bored in the middle of a long file position bias is real . A scanner that "usually checks most of the code" is not a scanner. Wall 3: They invent. In my other project, an AI chatbot invented products that didn't exist in the store. An AI bug-hunter does the same thing with vulnerabilities — confident reports about code that isn't there. Now your security tool hallucinates CVEs. Great. If you've read my earlier article about the chatbot that lied to customers https://dev.to/alimafana/your-ai-is-lying-to-your-customers-and-prompt-engineering-wont-fix-it-5408 , you know where this goes: you can't prompt your way out of a structural problem. This is the same disease in a different body. Fine — so use deterministic tools. Pattern rules, taint tracking. Semgrep and CodeQL have done it for years. But rules have their own wall, and my flagged-but-safe method above IS that wall. A taint tracker follows data: user input enters at getParameter the source , travels through variables, and reaches executeUpdate the sink — the dangerous operation . Path exists → alarm. That's the whole trick, and it's a good trick. It's deterministic, fast, and it never gets bored. But look at the method again. The data DOES flow from source to sink. The taint tracker is not wrong about the flow. It's wrong about the meaning — the matches " 0-9 +" guard makes the flow harmless, and understanding that requires understanding what the code means , not just where the data goes . Rules can't read meaning. So real-world scanners over-flag, developers drown in false alarms, and — every security team knows this story — they stop reading the reports. The industry's own benchmark makes the point better than I can: the OWASP Benchmark 2,740 labeled Java test cases, the standard test set for Java scanners contains 701 cases in my scanner's categories that are deliberately designed to trick tools into false alarms. Nearly half the test set exists just So: Everyone's strength is the other one's weakness. You can see the answer coming. Police detectives don't sentence anyone. Judges don't collect evidence. The system splits the work on purpose: exhaustive, procedural search by one party; careful judgment of each individual case by another. That's the architecture: Deterministic rules do ALL the searching. The LLM only judges what the rules found. A memory of real-world bugs helps it judge better. Concretely, the pipeline looks like this: source code → Joern builds a Code Property Graph code → queryable data-flow graph → fixed taint queries find candidates same input = same output, always → slicer cuts a ~small snippet per candidate the evidence file → RAG fetches similar known CVEs past cases, from GitHub advisories → LLM judges ONE candidate at a time real bug? severity? fix? The detective is Joern https://joern.io — an open-source static analysis tool that turns code into a graph you can query. My whole detection layer is a small table of rules. Adding a vulnerability class is one row: case class VulnClass name: String, // "sql-injection" cwe: String, // "CWE-89" sinkNames: String, // which call is dangerous sinkMethodFullName: String, // narrowed to the right type sinkArg: Int, // which argument carries the taint 0 = any sinkReceiver: String = "" // fallback when the type can't be resolved val classes = List VulnClass "sql-injection", "CWE-89", "executeQuery|executeUpdate|executeLargeUpdate|execute|addBatch|prepareStatement|prepareCall|nativeSQL", ". java|javax \\.sql\\.. ", 1 , VulnClass "command-injection", "CWE-78", "exec", ". Runtime. ", 0 , VulnClass "command-injection", "CWE-78", "<init |command", ". ProcessBuilder. ", 0 , VulnClass "path-traversal", "CWE-22", "<init ", ". java\\.io\\. File|FileInputStream|FileOutputStream|FileReader|FileWriter|RandomAccessFile \\.<init . ", 0 , // ... 3 more rows JdbcTemplate SQLi, Paths.get, XSS // for each row: is there a data-flow path from source to sink? sink.reachableByFlows source Sources — getParameter , getHeader , getCookies and the rest of the HTTP request surface — are shared by every class, so they live in one regex above the table rather than being repeated on each row. One line — reachableByFlows — is the entire detector. It over-flags by design it flagged our digits-only method , and that's fine. Detectives are supposed to bring in every plausible suspect. Their job is to miss nothing. The judge is Gemma via Google's API , and this is the important part: it never sees the codebase. It sees one small slice of code — the few lines the { "reasoning": "≤ 3 sentences, generated FIRST, before the verdict", "confirmed": true, "severity": "Critical | High | Medium | Low CVSS bands ", "explanation": "the concrete risk", "fix": "corrected code" } Judging one small snippet is the task LLMs are genuinely good at. There's nothing to search, nothing to miss, nothing to invent — the evidence is the whole context. And because discovery is frozen, the same suspects go to the judge every run. The randomness is contained to the one layer where a second opinion is the point. One prompt detail that gets a full article later in the series: you have to actively stop the judge from agreeing with the detective. Tell a model "a scanner flagged this" and it wants to say yes. The prompt explicitly states that rejecting a finding is a correct and expected answer — otherwise the judge just rubber-stamps every arrest, and you've built an expensive echo. I test on a small OWASP-style sample set: 8 methods — 4 real vulnerabilities across SQL injection, command injection, path traversal, and XSS; 2 methods that are flagged by the rules but actually sanitized the digits-only method above is one ; and 2 genuinely safe methods. | Stage | Result | |---|---| | Discovery rules | found all 6 flows: the 4 real + the 2 sanitized traps. Silent on the 2 safe methods. | | Judge an earlier, smaller Gemma | 6/6 correct — confirmed the 4 real with CVSS severities , rejected both sanitized false alarms | That rejection line is the thesis in one row. The deterministic layer cannot recognize a sanitizer. The LLM can. Division of labor works. And the architecture scales further than I expected for something run from a laptop: on the full OWASP Benchmark, Joern built the graph over 2,766 files in 100 seconds and ran the full query library — all seven rows — in Then I ran it against the real thing — the full OWASP Benchmark, 1,478 labeled cases in my four categories, scored against Semgrep and CodeQL by one program reading everyone's SARIF: | tool | precision | recall | F1 | |---|---|---|---| | CodeQL | 0.65 | 1.00 | 0.78 | | Semgrep | 0.60 | 0.86 | 0.70 | | my rules alone | 0.56 | 1.00 | 0.72 | my rules + the judge sample-derived | 0.67 | 0.98 | ~0.79 | The judge removed 52% of the false alarms and cost 2% of the real bugs in the latest verified run. One note before you check my arithmetic: the judged row is measured inside a 200-case stratified sample that deliberately over-samples false alarms discovery precision there starts at 0.51, not 0.56 — so 0.67 is a measured number, not "614 minus 52%". That caveat ships with the number. XSS went from 0.50 to 0.78 precision — those are the ESAPI.encodeForHTML traps, exactly the "is this actually safe?" question a taint engine can't answer. My rules find every one of the 777 real vulnerabilities — the same recall as CodeQL, from that seven-row table. But read the precision column before you get excited: CodeQL reaches perfect recall with 427 false positives, and I need 614 to do it. The difference is years of hand-built sanitizer knowledge that a seven-row table doesn't have. Honesty section, because this is an engineering log and not a launch post: CodeQL beats my rules layer on F1, 0.78 to 0.72, and I'd rather say that than hunt for a framing where I win. The claim I can defend is narrower and more interesting — a seven-row rule table plus a 31B open model lands in the same F1 range as a mature commercial engine, and the judge is what closes the precision gap. 1. Give the probabilistic system the judgment task, never the coverage task. Any job where "misses nothing, same answer every time" matters — searching, enumerating, auditing — belongs to deterministic code. The LLM gets the job where meaning matters and the input is small. This split applies way beyond security: it's the same reason my chatbot searches the product database with SQL and only lets the LLM phrase the answer. 2. If you can't re-run it, you can't measure it. Deterministic discovery means every experiment is repeatable: same code in, same candidates out, and any change in results traces to the one layer I changed. The moment discovery is probabilistic, comparisons against other tools become vibes. 3. Over-flagging is fine if someone competent reviews the flags. I stopped trying to make the rules smart. Rules that catch everything + a judge that understands meaning beat clever rules with no judge. Design each layer to fail in the direction the next layer can fix. The academic version of this argument exists too — a 2025 paper called LLMxCPG arXiv 2507.16585 pairs Code Property Graphs with LLM judgment the same way, which told me the instinct wasn't just mine. The part I hadn't seen done in public is the one above: running it against the incumbents on the industry's own benchmark and publishing every number, including the losses. This is the first article in a series where I do that in the open. Next: the three words that explain almost every injection bug — source, sink, and taint — written for anyone who's never done security work. After that, the false-alarm problem in detail, and what happened when I gave the same 200 code snippets to three different AI judges — including why the frontier model didn't win. I'm Ali Afana — AI builder and security researcher, writing from Gaza. I build systems in public, measure them against ground truth, and keep the receipts. This scanner is one project on a longer road — follow for what comes next.