CVE reachability with an LLM and a code graph

A developer built vex-engine, a GitLab CI pipeline that uses an LLM on Amazon Bedrock to read each CVE and OWASP dep-scan's code property graph to locate the vulnerable symbol in source, then commits a CycloneDX VEX verdict so Dependency-Track surfaces only reachable vulnerabilities. On one project, roughly seven or eight of ten critical CVEs were found unreachable from the code, though the developer cautions the ratio is anecdotal. The design replaces regex-based code scanning with graph queries to avoid false negatives that could hide a real vulnerability behind a confident not_affected verdict.

Asking a language model whether a CVE affects your code is easy. Deciding when not to believe the answer is the hard part. On one real project we started with a hundred critical CVEs and went through them by hand. Seven or eight out of ten could not be reached from our code. That ratio comes from one codebase, so treat it as an anecdote. Anyone who runs a scanner knows the effect anyway: when everything is red, nobody opens the dashboard, and each CVE costs a developer an afternoon of reading advisories and grepping code. vex-engine is the pipeline I built to give that afternoon back. It runs on GitLab CI, reads each CVE with an LLM on Amazon Bedrock, looks for it in the code with the graph that OWASP dep-scan builds, and commits a CycloneDX VEX verdict to the repository. Dependency-Track then shows only the CVEs that need a person. There are two ways to get this wrong. A false positive costs a developer some time. A false negative hides a real vulnerability behind a confident not affected , and preventing that is the reason for most of the design. So I follow one example through the pipeline, a Pillow CVE in an image decoder, and at every stage I ask the same question: could this stage hide a real vulnerability, and what stops it? The code, the CI template and the lab app are on GitLab https://gitlab.com/argomauro/vex-engine . Every snippet below comes from that repository. On every push, Trivy reads requirements.txt and writes a CycloneDX SBOM. Dependency-Track matches each component against NVD, GitHub and OSV and returns the CVEs. In parallel, OWASP dep-scan 6.3 reads the source and builds a code property graph with its atom engine. The analysis then takes one CVE at a time through three modules: The verdict goes into vex.json . Exploitable CVEs open one GitLab issue per component, and people close issues with labels that flow back into the VEX. The first version also had three phases, but the middle one was a Scanner. The Strategist wrote search patterns, the Scanner ran them as regular expressions over the files that imported the library, and the Auditor received thirty lines above and below every match. Noise went down. Then it met this: config = yaml.safe load open "settings.yml" safe data = yaml.load request.body vulnerable A text search finds both lines. The Auditor gets two blocks of code with no idea where request.body comes from or who calls the surrounding function. It has to guess, and on a bad day it guesses not affected . I replaced the Scanner with a Locator that queries a code graph. The Scanner stayed in the codebase with two smaller jobs, covered below. The division of labour stayed the same, and the rest of the post builds on it: the LLM reads the CVE at the start and judges facts at the end. The graph does the looking. A CVE is written for people. "Heap overflow in the TIFF decoder when processing crafted images" does not tell you which function in your code triggers it, and no database does either. NVD used to add structured data such as scores and affected configurations to every entry; in April NIST announced it would stop enriching every CVE. Reading an advisory and mapping it to code used to need someone with security experience, every time. The Strategist does that reading. It receives the CVE description and the package name and returns a plan as strict JSON. For CVE-2017-18342 PyYAML before 5.1, yaml.load can execute arbitrary code on untrusted data the plan looks like this, with illustrative values: { "trigger type": "api call", "vulnerable symbols": "yaml.load", "yaml.load all" , "conditions": "the argument comes from untrusted input and no SafeLoader is passed", "safe alternatives": "yaml.safe load", "yaml.safe load all" , "import aliases": "yaml" , "attack vector": "a YAML document with a python/object tag executes code when loaded" } Each field has a consumer downstream. The Locator uses vulnerable symbols and import aliases Pillow is imported as PIL , PyYAML as yaml . The Auditor uses conditions , attack vector and safe alternatives . This is the first place a real vulnerability can disappear. If the Strategist names the wrong function, the graph looks for the wrong thing. This part of the prompt guards against it: - vulnerable symbols: ALL the PUBLIC functions/classes an application may call that lead to the vulnerable code, fully qualified from the TOP-LEVEL import name. If the bug is inside an internal decoder or C module e.g. TiffDecode.c , name the public entry points that reach it e.g. "Image.open" for image decoders . If it is in a class used internally by convenience functions, list those too e.g. a bug in requests.Session redirects: "requests.get", "requests.post", ... . Missing an entry point hides a real vulnerability, so when unsure list more, not fewer. In the Pillow example the bug lives in a C decoder that no application calls directly. Applications call Image.open , which reaches the decoder when it opens a TIFF. The prompt asks the Strategist to make that jump, from where the bug is to where your code touches it. The plan also says how the bug fires: api call : the application must call a specific function. framework implicit : the library acts on its own. A CVE in Django's multipart parser fires on every upload before your view runs, and your code never names the parser. config : a setting turns the bug on. The Strategist picks the type from the CVE text, and the type decides which question the Locator asks. Nothing uses the answer before validation. The JSON must parse, the trigger type must be one of the three values, and symbol names are normalised. If Bedrock is unreachable, a keyword heuristic produces a fallback plan and the pipeline marks it as heuristic, because a guessed plan must never clear a CVE. Plans are cached per run, so each CVE/package pair goes to the model once. A wrong symbol list is the cheapest way to hide a vulnerability. Ask for more entry points, never fewer, and record where each plan came from. The Locator uses no model. It reads the graph that OWASP dep-scan https://github.com/owasp-dep-scan/dep-scan builds with atom, which combines three views of the same code. The syntax tree knows that Image in your file is Pillow's module. The call graph knows who calls whom. The data flow knows where a value travels. dep-scan exports the graph as two JSON slices. usages lists every call with its arguments and line. reachables lists flows that start from external input, such as an HTTP request. The CI job pins the image, because the slice format changes between atom versions, and writes a marker file only when the slices are complete: vex reachability: stage: analyze image: name: ghcr.io/owasp-dep-scan/dep-scan:v6.3.0 entrypoint: "" allow failure: true script: - | if depscan -i "${VEX APP SRC}" -o "${CI PROJECT DIR}/depscan-out" -t python \ --reachability-analyzer SemanticReachability \ --explanation-mode EndpointsAndReachables \ --vdb-scope app --no-banner \ && -s depscan-out/python-usages.slices.json ; then touch depscan-out/.complete fi dep-scan exits 0 even when it produces no slices. Partial slices would report "library not used" for every file that failed to parse, and the analysis would close CVEs it never looked at. Without .complete , the Locator treats the graph as missing. Matching is one small function. atom writes resolved calls such as PIL.py:<module .Image.open . The Locator splits them into segments and compares from the end: php def symbol matches qualified: List str , symbol: str - bool: sym = s for s in norm symbol .replace " ", "" .split "." if s if not sym or len sym len qualified : return False return qualified -len sym : == sym PIL.Image.open matches pil, image, open . yaml.load matches yaml.py:<module .load and never safe load , because the last segment differs. That fixes the case that sank the Scanner. Pin the dep-scan image and check that the slices exist. A graph that failed silently looks exactly like a codebase that never uses the library. The graph can give four answers, and each has a fixed next step. For every call site the Locator collects the file and line, the origin of each argument, the external-input flows that cross the enclosing function, every path from an entry point to the call, and the source of each function on those paths. All of it goes to the Auditor. This is the Pillow case: Image.open is called at line 17 of upload avatar . This looks like good news, and it is the outcome most likely to hide a real problem, because the Strategist can still miss an entry point. Take a CVE in the redirect handling of requests.Session . The Strategist names requests.Session . The application only does this: r = requests.get user supplied url requests.get builds a Session internally, so the vulnerable code runs. Stopping at "not called" would be wrong. Before trusting it, the Locator runs three checks, one for each way "not called" could be false: | Check | Fails when | Why it matters | |---|---|---| | The symbols came from the LLM | Bedrock was down and the keyword heuristic produced them | a guessed list proves nothing | | They are real, callable APIs | the plan names TiffDecode.c , a private imaging module or a decoder plugin | no application calls a C file, so "not called" is trivially true | | A text search agrees | grep finds ImageMath.eval where the graph saw nothing | the graph has a hole: an unparsed file, an alias, a getattr | Any failure sends the CVE to in triage . If all three pass, the Auditor receives every use of the library and one question: does any of these calls reach the vulnerable code internally? For framework implicit and config there is no symbol to look for. If the library is imported, the Locator collects every place the application uses it and the Auditor decides whether the vulnerable feature is in use. On the lab app it sees upload avatar reading request.FILES , so multipart uploads are in use. If the library is not imported anywhere, the CVE goes straight to in triage . A server, a plugin or a string in the settings can load a package without any import in your code. If dep-scan failed or the language is not Python, the old Scanner runs. If it finds uses, the Auditor judges. If it finds nothing, the CVE goes to in triage , because without the graph "not found in the text" proves nothing. The relationship between the graph and the regex comes down to one rule: If the graph says "called", believe it. If it says "not called" or "not imported", the text has to agree. When they disagree, doubt wins. For an api call CVE, "never imported" sounds conclusive. It isn't, because the graph only sees your code. If Django uses sqlparse internally, a sqlparse CVE can be reached through Django, and the graph never looks inside Django. An SBOM generated from requirements.txt has no dependency tree. The Locator rebuilds one from the requires dist metadata on PyPI, breadth-first from the SBOM's components. On the lab app that is 34 packages in about two seconds. PyPI cannot tell you whether Django calls the vulnerable function in sqlparse. It only tells you that Django declares a dependency on it. That is enough to know when a CVE must stay open: | Library | PyPI tree | Verdict | |---|---|---| | jinja2 | direct, nothing else requires it | not affected if no text search finds an import | | sqlparse | required by Django | in triage | | private package | PyPI returns 404 | in triage | The jinja2 row is the only case the Locator closes on its own, without calling the LLM. Before it does, a regex looks for any import of the package, including dynamic ones through importlib or import . Optional extras count as dependents, but the walk does not traverse them. Following every extra makes every library transitive Pillow → docs extra → sphinx → jinja2 and nothing could ever be closed. The known trade-off: a package reached only through another package's extra, such as requests security → pyOpenSSL → cryptography, can look direct. The graph sees your code. PyPI sees what your dependencies declare. Neither one is allowed to clear a CVE on a guess. The Auditor receives the Strategist's plan, the CVE description and a block of evidence built from the slices. The prompt tells it what kind of material it is reading: EVIDENCE FROM THE CODE GRAPH OWASP dep-scan . For every call site of the vulnerable API you get: the arguments' origin, the external-input flows that cross the enclosing function, every call path from an entrypoint, and the code of every function on those paths. These are facts from data-flow analysis, not guesses: the vulnerable API IS called at these lines. For each call site decide: 1. Can attacker-controlled data HTTP request, uploaded file, message, CLI arg reach the call through one of the paths? Follow the arguments' origin. 2. Are the trigger conditions met arguments, options, file format, settings ? 3. Is there validation or a safe alternative that blocks the attack? The CVE is "affected" if at least one call site is exploitable. A rule engine can't answer these questions. Is there validation before the call? Is the argument a constant or an upload? Does the file format match the CVE? Is yaml.load called with a SafeLoader ? A person answers in seconds when looking at the right snippet. The graph's job is to put that snippet, and the path that leads to it, in front of the model. The evidence is capped so prompts stay small: at most 8 call sites per CVE, 5 paths per call site, call depth 8, and 80 lines per function. That is enough to follow a request from a Django view to a library call in a typical web app. The Auditor answers with strict JSON: a verdict that is exactly affected or not affected , a confidence between 0 and 1, a reasoning that cites the path, and the call sites it considers exploitable. The graph decides what the model sees. The model decides what it means. In case 2 above library used, vulnerable function not called the Auditor is asked whether requests.get reaches Session internally. Nothing in the evidence answers that. The engine passes no library source, no documentation and no web search, so the model answers from training data. For widely used libraries such as requests, Pillow, PyYAML and Django, that is usually right: the mapping between public functions and internal classes is stable and well documented. For a niche package, or a release newer than the model's training cutoff, it can be wrong. A smaller model knows less than a larger one. Two things contain this. The confidence threshold sends an unsure answer to in triage , which is what usually happens with a library the model barely knows. And every reasoning is written into the VEX and committed, where a reviewer can read and challenge it. A model that is confidently wrong passes the threshold; only the review catches it. The next step is to replace memory with facts: have atom also build a graph of the installed library and show the Auditor the real path from requests.get to Session . vex-engine does not do this yet. The final rule does not depend on the model: php Fail-closed: None doubt, error, low confidence - in triage, stays visible if is reachable is None: state = "in triage" else: state = "exploitable" if is reachable else "not affected" A verdict with confidence of 0.8 or more VEX MIN CONFIDENCE becomes exploitable or not affected . Everything else becomes in triage and stays on the dashboard for a person: a malformed answer, a lower confidence, a timeout, a missing package URL from Dependency-Track. One failure mode only shows up over time. An LLM is not deterministic, and on an ambiguous CVE it can flip between not affected and exploitable on every run, closing and reopening the same issue. The bot counts its own flips. After two VEX MAX AUTO FLIPS it stops touching the issue, leaves it open and labels it vex::needs-human . A human decision resets the counter. This rule is what makes a smaller model safe to use. I ran the whole pipeline with a 12B open model through Ollama on a laptop. It produced more in triage verdicts than the Bedrock model and no hidden errors. Worst case, vex-engine leaves you with the dashboard you already had. Back to the Pillow decoder CVE on the lab app old Pillow, PyYAML and Django, with insecure code written on purpose . The Strategist returns trigger type: api call with PIL.Image.open among the symbols, because that is the public function that reaches the decoder. The plan came from the model, and the symbol is a real API. The Locator finds Image.open in the graph and builds the evidence. Simplified: Call site 1: Image.open @ vuln demo.py:17 in upload avatar Vulnerable symbol: PIL.Image.open Arguments: img file <- request.FILES 'avatar' Call paths entrypoint - call : <module -- line 56 -- upload avatar -- line 17 -- Image.open Code of the functions involved: def upload avatar request : ... img file = request.FILES 'avatar' with Image.open img file as img: The Auditor does not have to find the call or trace the data. It only decides whether an attacker controls that file: it is a user upload, and nothing validates it before the decoder runs. Verdict affected , confidence above the threshold, call site vuln demo.py:17 . The result is a VEX entry marked exploitable and a GitLab issue for the Pillow component with the file, the line, the origin of the argument and the path from the entry point. The developer who opens it can go straight to the fix. A full run on the lab app looks like this. The app is vulnerable by design, so the exploitable share is far higher than in a real codebase: Of 80 CVEs, 40 are closed with a written reason, 6 stay visible for a person, and 34 exploitable ones are grouped into two issues, one for Pillow and one for Django. Every verdict is written to vex.json in the application's repository, using standard CycloneDX fields, and committed with skip ci only when something changed: { "id": "CVE-XXXX-YYYY", "analysis": { "state": "not affected", "justification": "code not reachable", "response": "will not fix" , "detail": "Vulnerability CVE-XXXX-YYYY analyzed by LLM Analyzer python on 2026-10-20. <Auditor reasoning " } } In the uploaded VEX, affects .ref must stay the Dependency-Track project UUID the BOM's metadata.component.bom-ref . If you rewrite it with the component's purl, Dependency-Track returns HTTP 200 and silently discards the verdict. The only trace is Unable to locate affected element in its logs. vex-engine keeps the purl in a separate property, vex:component-purl . Issues open only for exploitable CVEs, one per component. People close them with a label that says why, and the pipeline turns the label into a VEX decision: | Label | VEX written | Expiry | What the bot does | |---|---|---|---| | vex::fixed | nothing forced, it's a claim | none | next analysis checks the graph; still reachable → issue reopens | | vex::false-positive | false positive | 90 days | skipped until expiry, then back to in triage | | vex::mitigated | not affected + protected by mitigating control | 90 days | same | | vex::risk-accepted | exploitable + will not fix | 90 days | stays visibly exploitable | If VEX APPROVERS is set and someone else closes the issue, the bot reopens it. Two vex:: labels at once count as a conflict. A person reopening an issue revokes the decision. When a library is upgraded and its CVEs leave the SBOM, the issue closes by itself. A decision that never expires becomes a blind spot. Ninety days is a setting. The expiry itself is the point. A pipeline that runs only on push misses new CVEs on code nobody touches, and misses the decisions people make on issues. vex-engine runs on three triggers: VEX MODE=critical . vex:: label to a closed one, runs vex issue sync --issues-only : no scan, no LLM, Dependency-Track updated in about a minute. VEX MODE=full catches new CVEs on unchanged code. Two jobs write vex.json , so they share a GitLab resource group and run one at a time. Set its process mode to oldest first to keep them in order. A project enables everything by including one template: include: - project: 'demo-security/vex-engine' file: 'ci-template.yml' ref: main stages: scan, upload-bom, analyze, sync-vex The LLM is called in two places, the Strategist and the Auditor, and only during an analysis. The SBOM, Dependency-Track, the graph, the webhook and the nightly sync make no model calls. The Strategist runs once per CVE in scope; the Auditor runs only for CVEs the graph does not settle on its own. The default model is Amazon Nova 2 Lite. The task is narrow and structured read text, return JSON , so a small model is enough. Switching model is one variable, and the same prompts run locally through Ollama, in which case no code leaves the machine: LLM PROVIDER=bedrock or: ollama BEDROCK MODEL ID=eu.amazon.nova-2-lite-v1:0 default OLLAMA MODEL=gemma4:12b local option VEX MIN CONFIDENCE=0.8 A run costs cents on Bedrock and nothing locally. The real infrastructure cost is the graph: atom runs on the JVM and the dep-scan image is about 3.8 GB, so cache it on your runners. Each of these ends in in triage , never in not affected : | Component | Configuration | What it guarantees | |---|---|---| | Strategist prompt | all public entry points, more when unsure | the graph looks for every way in | | Fallback plans | marked as heuristic | a guessed plan never clears a CVE | | dep-scan job | pinned image, .complete marker | a failed graph is treated as missing | | Symbol matching | segment suffix match | yaml.load never matches safe load | | "Not called" checks | LLM origin, real API, regex agreement | a hole in the graph ends in triage | | PyPI tree | direct leaf only, extras not traversed | transitive libraries stay visible | | Auditor evidence | 8 sites, 5 paths, depth 8, 80 lines | small prompts with the right snippet | | Verdict rule | VEX MIN CONFIDENCE=0.8 , fail closed | doubt never hides a CVE | | Anti-oscillation | VEX MAX AUTO FLIPS=2 , vex::needs-human | an unstable verdict goes to a person | | VEX upload | keep affects .ref as the project UUID | Dependency-Track keeps the verdicts | | Labels | approvers, 90-day expiry | decisions are attributed and revisited | | Writers | shared resource group | concurrent jobs don't drop decisions | Everything here ran with OWASP dep-scan 6.3.0, Dependency-Track 4.13 and Amazon Nova 2 Lite on Bedrock, on Python projects. The slice format changes between atom versions, so expect the matching code to move. To run it on your own project: clone vex-engine https://gitlab.com/argomauro/vex-engine , point VEX APP SRC at your source, add the include: block above to your .gitlab-ci.yml , and start with LLM PROVIDER=ollama if you'd rather keep code on your machine for the first run. The lab app in the repo is the fastest way to see all four routing cases.