# CVE reachability with an LLM and a code graph > Source: > Published: 2026-10-30 13:10:12+00:00 Asking a language model whether a CVE affects your code is easy. Deciding when not to believe the answer is the hard part. On one real project we started with a hundred critical CVEs and went through them by hand. Seven or eight out of ten could not be reached from our code. That ratio comes from one codebase, so treat it as an anecdote. Anyone who runs a scanner knows the effect anyway: when everything is red, nobody opens the dashboard, and each CVE costs a developer an afternoon of reading advisories and grepping code. vex-engine is the pipeline I built to give that afternoon back. It runs on GitLab CI, reads each CVE with an LLM on Amazon Bedrock, looks for it in the code with the graph that OWASP dep-scan builds, and commits a CycloneDX VEX verdict to the repository. Dependency-Track then shows only the CVEs that need a person. There are two ways to get this wrong. A false positive costs a developer some time. A false negative hides a real vulnerability behind a confident `not_affected`, and preventing that is the reason for most of the design. So I follow one example through the pipeline, a Pillow CVE in an image decoder, and at every stage I ask the same question: **could this stage hide a real vulnerability, and what stops it?** The code, the CI template and the lab app are on [GitLab](https://gitlab.com/argomauro/vex-engine). Every snippet below comes from that repository. On every push, Trivy reads `requirements.txt` and writes a CycloneDX SBOM. Dependency-Track matches each component against NVD, GitHub and OSV and returns the CVEs. In parallel, OWASP dep-scan 6.3 reads the source and builds a code property graph with its atom engine. The analysis then takes one CVE at a time through three modules: The verdict goes into `vex.json`. Exploitable CVEs open one GitLab issue per component, and people close issues with labels that flow back into the VEX. The first version also had three phases, but the middle one was a Scanner. The Strategist wrote search patterns, the Scanner ran them as regular expressions over the files that imported the library, and the Auditor received thirty lines above and below every match. Noise went down. Then it met this: ``` config = yaml.safe_load(open("settings.yml")) # safe data = yaml.load(request.body) # vulnerable ``` A text search finds both lines. The Auditor gets two blocks of code with no idea where `request.body` comes from or who calls the surrounding function. It has to guess, and on a bad day it guesses `not_affected`. I replaced the Scanner with a Locator that queries a code graph. The Scanner stayed in the codebase with two smaller jobs, covered below. The division of labour stayed the same, and the rest of the post builds on it: the LLM reads the CVE at the start and judges facts at the end. The graph does the looking. A CVE is written for people. "Heap overflow in the TIFF decoder when processing crafted images" does not tell you which function in your code triggers it, and no database does either. NVD used to add structured data such as scores and affected configurations to every entry; in April NIST announced it would stop enriching every CVE. Reading an advisory and mapping it to code used to need someone with security experience, every time. The Strategist does that reading. It receives the CVE description and the package name and returns a plan as strict JSON. For CVE-2017-18342 (PyYAML before 5.1, `yaml.load()` can execute arbitrary code on untrusted data) the plan looks like this, with illustrative values: ``` { "trigger_type": "api_call", "vulnerable_symbols": ["yaml.load", "yaml.load_all"], "conditions": "the argument comes from untrusted input and no SafeLoader is passed", "safe_alternatives": ["yaml.safe_load", "yaml.safe_load_all"], "import_aliases": ["yaml"], "attack_vector": "a YAML document with a python/object tag executes code when loaded" } ``` Each field has a consumer downstream. The Locator uses `vulnerable_symbols` and `import_aliases` (Pillow is imported as `PIL`, PyYAML as `yaml`). The Auditor uses `conditions`, `attack_vector` and `safe_alternatives`. This is the first place a real vulnerability can disappear. If the Strategist names the wrong function, the graph looks for the wrong thing. This part of the prompt guards against it: ``` - vulnerable_symbols: ALL the PUBLIC functions/classes an application may call that lead to the vulnerable code, fully qualified from the TOP-LEVEL import name. If the bug is inside an internal decoder or C module (e.g. TiffDecode.c), name the public entry points that reach it (e.g. "Image.open" for image decoders). If it is in a class used internally by convenience functions, list those too (e.g. a bug in requests.Session redirects: "requests.get", "requests.post", ...). Missing an entry point hides a real vulnerability, so when unsure list more, not fewer. ``` In the Pillow example the bug lives in a C decoder that no application calls directly. Applications call `Image.open`, which reaches the decoder when it opens a TIFF. The prompt asks the Strategist to make that jump, from where the bug is to where your code touches it. The plan also says how the bug fires: `api_call`: the application must call a specific function.` framework_implicit`: the library acts on its own. A CVE in Django's multipart parser fires on every upload before your view runs, and your code never names the parser.`config`: a setting turns the bug on. The Strategist picks the type from the CVE text, and the type decides which question the Locator asks. Nothing uses the answer before validation. The JSON must parse, the trigger type must be one of the three values, and symbol names are normalised. If Bedrock is unreachable, a keyword heuristic produces a fallback plan and the pipeline marks it as heuristic, because a guessed plan must never clear a CVE. Plans are cached per run, so each CVE/package pair goes to the model once. A wrong symbol list is the cheapest way to hide a vulnerability. Ask for more entry points, never fewer, and record where each plan came from. The Locator uses no model. It reads the graph that [OWASP dep-scan](https://github.com/owasp-dep-scan/dep-scan) builds with atom, which combines three views of the same code. The syntax tree knows that `Image` in your file is Pillow's module. The call graph knows who calls whom. The data flow knows where a value travels. dep-scan exports the graph as two JSON slices. `usages` lists every call with its arguments and line. `reachables` lists flows that start from external input, such as an HTTP request. The CI job pins the image, because the slice format changes between atom versions, and writes a marker file only when the slices are complete: ``` vex_reachability: stage: analyze image: name: ghcr.io/owasp-dep-scan/dep-scan:v6.3.0 entrypoint: [""] allow_failure: true script: - | if depscan -i "${VEX_APP_SRC}" -o "${CI_PROJECT_DIR}/depscan-out" -t python \ --reachability-analyzer SemanticReachability \ --explanation-mode EndpointsAndReachables \ --vdb-scope app --no-banner \ && [ -s depscan-out/python-usages.slices.json ]; then touch depscan-out/.complete fi ``` dep-scan exits 0 even when it produces no slices. Partial slices would report "library not used" for every file that failed to parse, and the analysis would close CVEs it never looked at. Without `.complete`, the Locator treats the graph as missing. Matching is one small function. atom writes resolved calls such as `PIL.py:.Image.open`. The Locator splits them into segments and compares from the end: ``` php def symbol_matches(qualified: List[str], symbol: str) -> bool: sym = [s for s in _norm(symbol).replace("()", "").split(".") if s] if not sym or len(sym) > len(qualified): return False return qualified[-len(sym):] == sym ``` `PIL.Image.open` matches `[pil, image, open]`. `yaml.load` matches `yaml.py:.load` and never `safe_load`, because the last segment differs. That fixes the case that sank the Scanner. Pin the dep-scan image and check that the slices exist. A graph that failed silently looks exactly like a codebase that never uses the library. The graph can give four answers, and each has a fixed next step. For every call site the Locator collects the file and line, the origin of each argument, the external-input flows that cross the enclosing function, every path from an entry point to the call, and the source of each function on those paths. All of it goes to the Auditor. This is the Pillow case: `Image.open` is called at line 17 of `upload_avatar`. This looks like good news, and it is the outcome most likely to hide a real problem, because the Strategist can still miss an entry point. Take a CVE in the redirect handling of `requests.Session`. The Strategist names `requests.Session`. The application only does this: ``` r = requests.get(user_supplied_url) ``` `requests.get` builds a `Session` internally, so the vulnerable code runs. Stopping at "not called" would be wrong. Before trusting it, the Locator runs three checks, one for each way "not called" could be false: | Check | Fails when | Why it matters | |---|---|---| | The symbols came from the LLM | Bedrock was down and the keyword heuristic produced them | a guessed list proves nothing | | They are real, callable APIs | the plan names `TiffDecode.c` , a private`_imaging` module or a decoder plugin | no application calls a C file, so "not called" is trivially true | | A text search agrees | grep finds `ImageMath.eval(` where the graph saw nothing | the graph has a hole: an unparsed file, an alias, a `getattr` | Any failure sends the CVE to `in_triage`. If all three pass, the Auditor receives every use of the library and one question: does any of these calls reach the vulnerable code internally? For `framework_implicit` and `config` there is no symbol to look for. If the library is imported, the Locator collects every place the application uses it and the Auditor decides whether the vulnerable feature is in use. On the lab app it sees `upload_avatar` reading `request.FILES`, so multipart uploads are in use. If the library is not imported anywhere, the CVE goes straight to `in_triage`. A server, a plugin or a string in the settings can load a package without any import in your code. If dep-scan failed or the language is not Python, the old Scanner runs. If it finds uses, the Auditor judges. If it finds nothing, the CVE goes to `in_triage`, because without the graph "not found in the text" proves nothing. The relationship between the graph and the regex comes down to one rule: If the graph says "called", believe it. If it says "not called" or "not imported", the text has to agree. When they disagree, doubt wins. For an `api_call` CVE, "never imported" sounds conclusive. It isn't, because the graph only sees your code. If Django uses sqlparse internally, a sqlparse CVE can be reached through Django, and the graph never looks inside Django. An SBOM generated from `requirements.txt` has no dependency tree. The Locator rebuilds one from the `requires_dist` metadata on PyPI, breadth-first from the SBOM's components. On the lab app that is 34 packages in about two seconds. PyPI cannot tell you whether Django calls the vulnerable function in sqlparse. It only tells you that Django declares a dependency on it. That is enough to know when a CVE must stay open: | Library | PyPI tree | Verdict | |---|---|---| | jinja2 | direct, nothing else requires it | `not_affected` if no text search finds an import | | sqlparse | required by Django | `in_triage` | | private package | PyPI returns 404 | `in_triage` | The jinja2 row is the only case the Locator closes on its own, without calling the LLM. Before it does, a regex looks for any import of the package, including dynamic ones through `importlib` or `__import__`. Optional extras count as dependents, but the walk does not traverse them. Following every extra makes every library transitive (Pillow → `docs` extra → sphinx → jinja2) and nothing could ever be closed. The known trade-off: a package reached only through another package's extra, such as `requests[security]` → pyOpenSSL → cryptography, can look direct. The graph sees your code. PyPI sees what your dependencies declare. Neither one is allowed to clear a CVE on a guess. The Auditor receives the Strategist's plan, the CVE description and a block of evidence built from the slices. The prompt tells it what kind of material it is reading: ``` EVIDENCE FROM THE CODE GRAPH (OWASP dep-scan). For every call site of the vulnerable API you get: the arguments' origin, the external-input flows that cross the enclosing function, every call path from an entrypoint, and the code of every function on those paths. These are facts from data-flow analysis, not guesses: the vulnerable API IS called at these lines. For each call site decide: 1. Can attacker-controlled data (HTTP request, uploaded file, message, CLI arg) reach the call through one of the paths? Follow the arguments' origin. 2. Are the trigger conditions met (arguments, options, file format, settings)? 3. Is there validation or a safe alternative that blocks the attack? The CVE is "affected" if at least one call site is exploitable. ``` A rule engine can't answer these questions. Is there validation before the call? Is the argument a constant or an upload? Does the file format match the CVE? Is `yaml.load` called with a `SafeLoader`? A person answers in seconds when looking at the right snippet. The graph's job is to put that snippet, and the path that leads to it, in front of the model. The evidence is capped so prompts stay small: at most 8 call sites per CVE, 5 paths per call site, call depth 8, and 80 lines per function. That is enough to follow a request from a Django view to a library call in a typical web app. The Auditor answers with strict JSON: a verdict that is exactly `affected` or `not_affected`, a confidence between 0 and 1, a reasoning that cites the path, and the call sites it considers exploitable. The graph decides what the model sees. The model decides what it means. In case 2 above (library used, vulnerable function not called) the Auditor is asked whether `requests.get` reaches `Session` internally. Nothing in the evidence answers that. The engine passes no library source, no documentation and no web search, so the model answers from training data. For widely used libraries such as requests, Pillow, PyYAML and Django, that is usually right: the mapping between public functions and internal classes is stable and well documented. For a niche package, or a release newer than the model's training cutoff, it can be wrong. A smaller model knows less than a larger one. Two things contain this. The confidence threshold sends an unsure answer to `in_triage`, which is what usually happens with a library the model barely knows. And every reasoning is written into the VEX and committed, where a reviewer can read and challenge it. A model that is confidently wrong passes the threshold; only the review catches it. The next step is to replace memory with facts: have atom also build a graph of the installed library and show the Auditor the real path from `requests.get` to `Session`. vex-engine does not do this yet. The final rule does not depend on the model: ``` php # Fail-closed: None (doubt, error, low confidence) -> in_triage, stays visible if is_reachable is None: state = "in_triage" else: state = "exploitable" if is_reachable else "not_affected" ``` A verdict with confidence of 0.8 or more (`VEX_MIN_CONFIDENCE`) becomes `exploitable` or `not_affected`. Everything else becomes `in_triage` and stays on the dashboard for a person: a malformed answer, a lower confidence, a timeout, a missing package URL from Dependency-Track. One failure mode only shows up over time. An LLM is not deterministic, and on an ambiguous CVE it can flip between `not_affected` and `exploitable` on every run, closing and reopening the same issue. The bot counts its own flips. After two (`VEX_MAX_AUTO_FLIPS`) it stops touching the issue, leaves it open and labels it `vex::needs-human`. A human decision resets the counter. This rule is what makes a smaller model safe to use. I ran the whole pipeline with a 12B open model through Ollama on a laptop. It produced more `in_triage` verdicts than the Bedrock model and no hidden errors. Worst case, vex-engine leaves you with the dashboard you already had. Back to the Pillow decoder CVE on the lab app (old Pillow, PyYAML and Django, with insecure code written on purpose). The Strategist returns `trigger_type: api_call` with `PIL.Image.open` among the symbols, because that is the public function that reaches the decoder. The plan came from the model, and the symbol is a real API. The Locator finds `Image.open` in the graph and builds the evidence. Simplified: ``` Call site 1: Image.open @ vuln_demo.py:17 (in upload_avatar) Vulnerable symbol: PIL.Image.open Arguments: img_file <- request.FILES['avatar'] Call paths (entrypoint -> call): --(line 56)--> upload_avatar --(line 17)--> Image.open Code of the functions involved: def upload_avatar(request): ... img_file = request.FILES['avatar'] with Image.open(img_file) as img: ``` The Auditor does not have to find the call or trace the data. It only decides whether an attacker controls that file: it is a user upload, and nothing validates it before the decoder runs. Verdict `affected`, confidence above the threshold, call site `vuln_demo.py:17`. The result is a VEX entry marked `exploitable` and a GitLab issue for the Pillow component with the file, the line, the origin of the argument and the path from the entry point. The developer who opens it can go straight to the fix. A full run on the lab app looks like this. The app is vulnerable by design, so the exploitable share is far higher than in a real codebase: Of 80 CVEs, 40 are closed with a written reason, 6 stay visible for a person, and 34 exploitable ones are grouped into two issues, one for Pillow and one for Django. Every verdict is written to `vex.json` in the application's repository, using standard CycloneDX fields, and committed with `[skip ci]` only when something changed: ``` { "id": "CVE-XXXX-YYYY", "analysis": { "state": "not_affected", "justification": "code_not_reachable", "response": ["will_not_fix"], "detail": "Vulnerability CVE-XXXX-YYYY analyzed by LLM Analyzer (python) on 2026-10-20. " } } ``` In the uploaded VEX, `affects[].ref` must stay the Dependency-Track project UUID (the BOM's `metadata.component.bom-ref`). If you rewrite it with the component's purl, Dependency-Track returns HTTP 200 and silently discards the verdict. The only trace is `Unable to locate affected element` in its logs. vex-engine keeps the purl in a separate property, `vex:component-purl`. Issues open only for exploitable CVEs, one per component. People close them with a label that says why, and the pipeline turns the label into a VEX decision: | Label | VEX written | Expiry | What the bot does | |---|---|---|---| | `vex::fixed` | nothing forced, it's a claim | none | next analysis checks the graph; still reachable → issue reopens | | `vex::false-positive` | `false_positive` | 90 days | skipped until expiry, then back to `in_triage` | | `vex::mitigated` | `not_affected` +`protected_by_mitigating_control` | 90 days | same | | `vex::risk-accepted` | `exploitable` +`will_not_fix` | 90 days | stays visibly exploitable | If `VEX_APPROVERS` is set and someone else closes the issue, the bot reopens it. Two `vex::` labels at once count as a conflict. A person reopening an issue revokes the decision. When a library is upgraded and its CVEs leave the SBOM, the issue closes by itself. A decision that never expires becomes a blind spot. Ninety days is a setting. The expiry itself is the point. A pipeline that runs only on push misses new CVEs on code nobody touches, and misses the decisions people make on issues. vex-engine runs on three triggers: `VEX_MODE=critical`).` vex::` label to a closed one, runs `vex_issue_sync --issues-only`: no scan, no LLM, Dependency-Track updated in about a minute.`VEX_MODE=full`) catches new CVEs on unchanged code. Two jobs write `vex.json`, so they share a GitLab resource group and run one at a time. Set its process mode to `oldest_first` to keep them in order. A project enables everything by including one template: ``` include: - project: 'demo-security/vex-engine' file: 'ci-template.yml' ref: main stages: [scan, upload-bom, analyze, sync-vex] ``` The LLM is called in two places, the Strategist and the Auditor, and only during an analysis. The SBOM, Dependency-Track, the graph, the webhook and the nightly sync make no model calls. The Strategist runs once per CVE in scope; the Auditor runs only for CVEs the graph does not settle on its own. The default model is Amazon Nova 2 Lite. The task is narrow and structured (read text, return JSON), so a small model is enough. Switching model is one variable, and the same prompts run locally through Ollama, in which case no code leaves the machine: ``` LLM_PROVIDER=bedrock # or: ollama BEDROCK_MODEL_ID=eu.amazon.nova-2-lite-v1:0 # default OLLAMA_MODEL=gemma4:12b # local option VEX_MIN_CONFIDENCE=0.8 ``` A run costs cents on Bedrock and nothing locally. The real infrastructure cost is the graph: atom runs on the JVM and the dep-scan image is about 3.8 GB, so cache it on your runners. Each of these ends in `in_triage`, never in `not_affected`: | Component | Configuration | What it guarantees | |---|---|---| | Strategist prompt | all public entry points, more when unsure | the graph looks for every way in | | Fallback plans | marked as heuristic | a guessed plan never clears a CVE | | dep-scan job | pinned image, `.complete` marker | a failed graph is treated as missing | | Symbol matching | segment suffix match | `yaml.load` never matches`safe_load` | | "Not called" checks | LLM origin, real API, regex agreement | a hole in the graph ends in triage | | PyPI tree | direct leaf only, extras not traversed | transitive libraries stay visible | | Auditor evidence | 8 sites, 5 paths, depth 8, 80 lines | small prompts with the right snippet | | Verdict rule | `VEX_MIN_CONFIDENCE=0.8` , fail closed | doubt never hides a CVE | | Anti-oscillation | `VEX_MAX_AUTO_FLIPS=2` ,`vex::needs-human` | an unstable verdict goes to a person | | VEX upload | keep `affects[].ref` as the project UUID | Dependency-Track keeps the verdicts | | Labels | approvers, 90-day expiry | decisions are attributed and revisited | | Writers | shared resource group | concurrent jobs don't drop decisions | Everything here ran with OWASP dep-scan 6.3.0, Dependency-Track 4.13 and Amazon Nova 2 Lite on Bedrock, on Python projects. The slice format changes between atom versions, so expect the matching code to move. To run it on your own project: clone [vex-engine](https://gitlab.com/argomauro/vex-engine), point `VEX_APP_SRC` at your source, add the `include:` block above to your `.gitlab-ci.yml`, and start with `LLM_PROVIDER=ollama` if you'd rather keep code on your machine for the first run. The lab app in the repo is the fastest way to see all four routing cases.