100% vuln detection wasn't enough: measuring whether AI respects the patch

This is a submission for the Kaggle Benchmarking Challenge

AI models are often like over-eager alarms. Show them a dangerous word in code — eval, system(, a raw SQL concat — and they scream “vulnerability!” nearly every time. Add the lock one line up, and many cheaper models still scream: they recognized the scary token, they did not read the fix.

That failure mode is expensive. Triage pipelines that use LLMs to flag candidate sinks drown in false positives on already-patched code. Public coding evals ask “did you find a bug?” They rarely ask “did you respect the fix?”

I built ART — Attacker-Reachable Sink Triage to measure that gap.

How it works: minimal-pair twins. Same function name, same identifiers, same shape — only the security control differs. Prompts get snippet + language only. Twin ids, gold labels, and rationales never enter the model context.

Here is a real pair from the set. The only difference is the fix — everything a token-matcher keys on ($_GET["id"], SELECT, the function name) is identical:

// twin_sql_php · gold = reachable_vuln
function process_user_data($conn) {
    $id = $_GET["id"];
    $sql = "SELECT * FROM users WHERE id = " . $id;   // attacker-controlled concat
    return mysqli_query($conn, $sql);
}
// twin_sql_php · gold = patched  (same shape, one control added)
function process_user_data($conn) {
    $id = (int)$_GET["id"];
    $stmt = mysqli_prepare($conn, "SELECT * FROM users WHERE id = ?");
    mysqli_stmt_bind_param($stmt, "i", $id);          // cast + prepared statement
    mysqli_stmt_execute($stmt);
    return mysqli_stmt_get_result($stmt);
}

A model that labels the second snippet reachable_vuln isn't a worse detector — it's a worse patch reader. Twin Gap captures exactly that: vuln accuracy − patched accuracy. Zero means the model respects fixes; positive means it over-flags patched code.

Core suite

Dataset: 8 twin pairs (SQLi, XSS, auth bypass, command injection, path traversal, LFI, insecure deserialization — PHP + Python) plus 6 safe/vacuous controls. N is intentionally small: one miss moves Twin Gap by 12.5%. This is a diagnostic probe, not a large-N ranking claim.

Why synthetic, not raw CVEs: so models cannot win by memorizing a write-up, and so each twin differs by one control. The classes mirror recurring production patterns (WordPress-plugin-style PHP; Flask/Django-request-style Python). Failing a patched twin here is meant to map to over-flagging a real fix in those ecosystems.

Ablations (personas + forced CoT) are public supporting tasks; the headline metric is label-triage.

Seven locked Community Benchmark models, chosen for tier × price × family coverage — not a single SOTA chase:

Roughly 50× cost span per full triage run (~$0.004 → ~$0.18). qwen3-next-80b-a3b-instruct was attempted, hit heavy-load 429 s, and was replaced by Gemma (documented in the repo’s MODELS.md).

Every model found every vulnerable twin (100% raw). That alone is useless — an alarm that never stops ringing doesn’t help. The useful question is whether the model calms down after the lock.

Numbers from art-label-triage v6 (adjudicated gold + production scoring):

Cost: Gemma and Gemini flash match pro-tier ART at roughly 1–4% of the cost. Heavy Pro does not beat Flash or Gemma on this probe.

1. The models corrected our gold.

All seven disagreed with two labels — in the same direction — and they were right. An escaped-input “safe” filler was really patched by our own prompt definition; a deser twin that replaced pickle.loads with json.loads was really safe. Those two items capped every model at ART 0.917 and invented our largest “failure” class. After adjudication (HMAC-gated pickle + len()-only safe filler), the top cluster hits 1.000 and remaining errors are patch misses under the frozen, adjudicated rubric.

That redesign is the point: a ceiling isn’t always model competence — sometimes it’s your key.

2. A 0.0 that wasn’t a capability.

Sonnet’s proof-marker stayed 0.0 across retries because the provider returned an empty completion (86 prompt tokens, empty message). Read the transcript before ranking a model on a single-shot cell.

3. Personas and CoT didn’t “fix” patch-respect.

Red-team persona did not systematically inflate overclaim. Forced data-flow CoT on the trap did not close Haiku’s gap (0.625 → 0.50).

basename("../../../etc/passwd") still traverses — it doesn’t (current_user_can works, then still labels reachable_vuln by shifting to a different risk (rewards.score (and the table above) — not the collection chart. Kaggle collection (required): Attacker-Reachable Sink Triage (ART)

Source (MIT): https://github.com/mziqudhd92/kaggle-art-benchmark

Core tasks

kaggle b t run art-label-triage -m gemini-3.5-flash --wait

Reproducibility: the dataset (dataset/items.jsonl) is frozen and versioned in the repo; gold labels are deterministic and scored by param_id, not answer order. scripts/validate_jsonl.py and scripts/test_scoring_alignment.py gate every push, and scripts/analyze_results.py regenerates the tables and charts above from the downloaded run artifacts. Every number here traces to a specific task version (label-triage v6).

Safety: synthetic snippets only; defensive triage research; no live targeting.

Credit: inspired by proof-over-speculation tooling (Iridium); this entry is a standalone Kaggle Community Benchmark under MIT.