My vulnerability scanner flagged this Java method as SQL injection:
String id = request.getParameter("id"); // user input
if (!id.matches("[0-9]+")) { // digits only, or throw
throw new IllegalArgumentException("id must be numeric");
}
String sql = "DELETE FROM products WHERE id = " + id;
stmt.executeUpdate(sql); // flagged as SQL injection
Look at line 2. The input must be pure digits or the method throws. You cannot
inject SQL through [0-9]+
. There is no attack here. The scanner flagged safe code.
Here's the part that sounds wrong: I designed it to do that. The false alarm is
not a bug in my scanner. It's the plan.
Let me explain, because this decision is the entire foundation of the scanner —
and I think it's the decision most people building "AI security tools" right now
are getting backwards.
I'm building an LLM-based vulnerability scanner — in public, like everything I
ship. The pitch is simple: AI writes a lot of code now, fast, often for people
who aren't security experts. That code ships with holes. Someone has to find
them.
So the obvious idea — the one I started with — is the one you've seen in a hundred
launch posts this year:
"Point an AI agent at your codebase. It reads everything and finds your
vulnerabilities."
I killed that idea before writing a line of code. Not because it doesn't sound
amazing. Because I couldn't defend it with numbers — and the reasons it can't be
measured are the same reasons you shouldn't trust it in production.
New to this?Two words carry the whole article. Asourceis where
untrusted input enters your program (request.getParameter
). Asinkis
where it becomes dangerous (executeUpdate
— running a database command). A
vulnerability is untrusted data reaching a sink without being cleaned on the
way. That's it; everything below builds on those two words.
Ask an LLM to audit a codebase and you hit three walls.
Wall 1: You get a different answer every run. LLMs are probabilistic. Same
repo, same prompt, run it twice — the bug lists don't match. Which run goes in
the security report? Which one do you benchmark? When a tool's output changes
between runs, you can't measure it, you can't compare it to Semgrep or CodeQL,
and you can't do science with it. You can only do demos.
Wall 2: Searching is the thing LLMs are bad at. Careful, exhaustive
enumeration over a large space — visit every file, check every call, miss
nothing — is exactly what LLMs don't do. They skim. They fixate. They get bored
in the middle of a long file (position bias is real). A scanner that "usually
checks most of the code" is not a scanner.
Wall 3: They invent. In my other project, an AI chatbot invented products
that didn't exist in the store. An AI bug-hunter does the same thing with
vulnerabilities — confident reports about code that isn't there. Now your
security tool hallucinates CVEs. Great.
If you've read my earlier article about the chatbot that lied to customers,
you know where this goes: you can't prompt your way out of a structural problem.
This is the same disease in a different body.
Fine — so use deterministic tools. Pattern rules, taint tracking. Semgrep and
CodeQL have done it for years.
But rules have their own wall, and my flagged-but-safe method above IS that wall.
A taint tracker follows data: user input enters at getParameter
(the source),
travels through variables, and reaches executeUpdate
(the sink — the
dangerous operation). Path exists → alarm. That's the whole trick, and it's a
good trick. It's deterministic, fast, and it never gets bored.
But look at the method again. The data DOES flow from source to sink. The taint
tracker is not wrong about the flow. It's wrong about the meaning — the
matches("[0-9]+")
guard makes the flow harmless, and understanding that
requires understanding what the code means, not just where the data goes.
Rules can't read meaning. So real-world scanners over-flag, developers drown in
false alarms, and — every security team knows this story — they stop reading the
reports. The industry's own benchmark makes the point better than I can: the
OWASP Benchmark (2,740 labeled Java test cases, the standard test set for Java
scanners) contains 701 cases in my scanner's categories that are deliberately designed to trick tools into false alarms. Nearly half the test set exists just
So:
Everyone's strength is the other one's weakness. You can see the answer coming.
Police detectives don't sentence anyone. Judges don't collect evidence. The
system splits the work on purpose: exhaustive, procedural search by one party;
careful judgment of each individual case by another.
That's the architecture:
Deterministic rules do ALL the searching. The LLM only judges what the rules
found. A memory of real-world bugs helps it judge better.
Concretely, the pipeline looks like this:
source code
→ Joern builds a Code Property Graph (code → queryable data-flow graph)
→ fixed taint queries find candidates (same input = same output, always)
→ slicer cuts a ~small snippet per candidate (the evidence file)
→ RAG fetches similar known CVEs (past cases, from GitHub advisories)
→ LLM judges ONE candidate at a time (real bug? severity? fix?)
The detective is Joern — an open-source static analysis tool
that turns code into a graph you can query. My whole detection layer is a small
table of rules. Adding a vulnerability class is one row:
case class VulnClass(
name: String, // "sql-injection"
cwe: String, // "CWE-89"
sinkNames: String, // which call is dangerous
sinkMethodFullName: String, // narrowed to the right type
sinkArg: Int, // which argument carries the taint (0 = any)
sinkReceiver: String = "" // fallback when the type can't be resolved
)
val classes = List(
VulnClass("sql-injection", "CWE-89",
"executeQuery|executeUpdate|executeLargeUpdate|execute|addBatch|prepareStatement|prepareCall|nativeSQL",
".*(java|javax)\\.sql\\..*", 1),
VulnClass("command-injection", "CWE-78", "exec", ".*Runtime.*", 0),
VulnClass("command-injection", "CWE-78", "<init>|command", ".*ProcessBuilder.*", 0),
VulnClass("path-traversal", "CWE-22", "<init>",
".*java\\.io\\.(File|FileInputStream|FileOutputStream|FileReader|FileWriter|RandomAccessFile)\\.<init>.*", 0),
// ... 3 more rows (JdbcTemplate SQLi, Paths.get, XSS)
)
// for each row: is there a data-flow path from source to sink?
sink.reachableByFlows(source)
(Sources — getParameter
, getHeader
, getCookies
and the rest of the HTTP
request surface — are shared by every class, so they live in one regex above the
table rather than being repeated on each row.)
One line — reachableByFlows
— is the entire detector. It over-flags by design
(it flagged our digits-only method), and that's fine. Detectives are supposed
to bring in every plausible suspect. Their job is to miss nothing.
The judge is Gemma (via Google's API), and this is the important part: it never sees the codebase. It sees one small slice of code — the few lines the
{
"reasoning": "≤ 3 sentences, generated FIRST, before the verdict",
"confirmed": true,
"severity": "Critical | High | Medium | Low (CVSS bands)",
"explanation": "the concrete risk",
"fix": "corrected code"
}
Judging one small snippet is the task LLMs are genuinely good at. There's
nothing to search, nothing to miss, nothing to invent — the evidence is the
whole context. And because discovery is frozen, the same suspects go to the
judge every run. The randomness is contained to the one layer where a second
opinion is the point.
(One prompt detail that gets a full article later in the series: you have to actively stop
the judge from agreeing with the detective. Tell a model "a scanner flagged
this" and it wants to say yes. The prompt explicitly states that rejecting a
finding is a correct and expected answer — otherwise the judge just rubber-stamps
every arrest, and you've built an expensive echo.)
I test on a small OWASP-style sample set: 8 methods — 4 real vulnerabilities
across SQL injection, command injection, path traversal, and XSS; 2 methods that
are flagged by the rules but actually sanitized (the digits-only method
above is one); and 2 genuinely safe methods.
| Stage | Result |
|---|---|
| Discovery (rules) | found all 6 flows: the 4 real + the 2 sanitized traps. Silent on the 2 safe methods. |
| Judge (an earlier, smaller Gemma) | |
| 6/6 correct — confirmed the 4 real (with CVSS severities), rejected both sanitized false alarms | |
That rejection line is the thesis in one row. The deterministic layer cannot
recognize a sanitizer. The LLM can. Division of labor works.
And the architecture scales further than I expected for something run from a
laptop: on the full OWASP Benchmark, Joern built the graph over 2,766 files in 100 seconds and ran the full query library — all seven rows — in
Then I ran it against the real thing — the full OWASP Benchmark, 1,478 labeled
cases in my four categories, scored against Semgrep and CodeQL by one program
reading everyone's SARIF:
| tool | precision | recall | F1 |
|---|---|---|---|
| CodeQL | 0.65 | 1.00 | 0.78 |
| Semgrep | 0.60 | 0.86 | 0.70 |
| my rules alone | 0.56 | 1.00 | |
| 0.72 | |||
| my rules + the judge (sample-derived) | |||
| 0.67 | |||
| 0.98 | ~0.79 |
The judge removed 52% of the false alarms and cost 2% of the real bugs
in the latest verified run. One note before you check my arithmetic: the judged
row is measured inside a 200-case stratified sample that deliberately
over-samples false alarms (discovery precision there starts at 0.51, not 0.56)
— so 0.67 is a measured number, not "614 minus 52%". That caveat ships with the
number.
XSS went from 0.50 to 0.78 precision — those are the ESAPI.encodeForHTML()
traps, exactly the "is this actually safe?" question a taint engine can't answer.
My rules find every one of the 777 real vulnerabilities — the same recall as
CodeQL, from that seven-row table. But read the precision column before you get
excited: CodeQL reaches perfect recall with 427 false positives, and I need 614
to do it. The difference is years of hand-built sanitizer knowledge that a
seven-row table doesn't have.
Honesty section, because this is an engineering log and not a launch post:
CodeQL beats my rules layer on F1, 0.78 to 0.72, and I'd rather say that than
hunt for a framing where I win. The claim I can defend is narrower and more
interesting — a seven-row rule table plus a 31B open model lands in the same F1
range as a mature commercial engine, and the judge is what closes the precision
gap.
1. Give the probabilistic system the judgment task, never the coverage task.
Any job where "misses nothing, same answer every time" matters — searching,
enumerating, auditing — belongs to deterministic code. The LLM gets the job
where meaning matters and the input is small. This split applies way beyond
security: it's the same reason my chatbot searches the product database with SQL
and only lets the LLM phrase the answer.
2. If you can't re-run it, you can't measure it. Deterministic discovery
means every experiment is repeatable: same code in, same candidates out, and any
change in results traces to the one layer I changed. The moment discovery is
probabilistic, comparisons against other tools become vibes.
3. Over-flagging is fine if someone competent reviews the flags. I stopped
trying to make the rules smart. Rules that catch everything + a judge that
understands meaning beat clever rules with no judge. Design each layer to fail
in the direction the next layer can fix.
The academic version of this argument exists too — a 2025 paper called LLMxCPG
(arXiv 2507.16585) pairs Code Property Graphs with LLM judgment the same way,
which told me the instinct wasn't just mine. The part I hadn't seen done in
public is the one above: running it against the incumbents on the industry's own
benchmark and publishing every number, including the losses.
This is the first article in a series where I do that in the open. Next: the
three words that explain almost every injection bug — source, sink, and taint —
written for anyone who's never done security work. After that, the false-alarm
problem in detail, and what happened when I gave the same 200 code snippets to
three different AI judges — including why the frontier model didn't win.
I'm Ali Afana — AI builder and security researcher, writing from Gaza. I build systems in public, measure them against ground truth, and keep the receipts. This scanner is one project on a longer road — follow for what comes next.