{"slug": "cve-reachability-with-an-llm-and-a-code-graph", "title": "CVE reachability with an LLM and a code graph", "summary": "A developer built vex-engine, a GitLab CI pipeline that uses an LLM on Amazon Bedrock to read each CVE and OWASP dep-scan's code property graph to locate the vulnerable symbol in source, then commits a CycloneDX VEX verdict so Dependency-Track surfaces only reachable vulnerabilities. On one project, roughly seven or eight of ten critical CVEs were found unreachable from the code, though the developer cautions the ratio is anecdotal. The design replaces regex-based code scanning with graph queries to avoid false negatives that could hide a real vulnerability behind a confident not_affected verdict.", "body_md": "Asking a language model whether a CVE affects your code is easy. Deciding when not to believe the answer is the hard part.\n\nOn one real project we started with a hundred critical CVEs and went through them by hand. Seven or eight out of ten could not be reached from our code. That ratio comes from one codebase, so treat it as an anecdote. Anyone who runs a scanner knows the effect anyway: when everything is red, nobody opens the dashboard, and each CVE costs a developer an afternoon of reading advisories and grepping code.\n\nvex-engine is the pipeline I built to give that afternoon back. It runs on GitLab CI, reads each CVE with an LLM on Amazon Bedrock, looks for it in the code with the graph that OWASP dep-scan builds, and commits a CycloneDX VEX verdict to the repository. Dependency-Track then shows only the CVEs that need a person.\n\nThere are two ways to get this wrong. A false positive costs a developer some time. A false negative hides a real vulnerability behind a confident `not_affected`, and preventing that is the reason for most of the design. So I follow one example through the pipeline, a Pillow CVE in an image decoder, and at every stage I ask the same question: **could this stage hide a real vulnerability, and what stops it?**\n\nThe code, the CI template and the lab app are on [GitLab](https://gitlab.com/argomauro/vex-engine). Every snippet below comes from that repository.\n\nOn every push, Trivy reads `requirements.txt` and writes a CycloneDX SBOM. Dependency-Track matches each component against NVD, GitHub and OSV and returns the CVEs. In parallel, OWASP dep-scan 6.3 reads the source and builds a code property graph with its atom engine. The analysis then takes one CVE at a time through three modules:\n\nThe verdict goes into `vex.json`. Exploitable CVEs open one GitLab issue per component, and people close issues with labels that flow back into the VEX.\n\nThe first version also had three phases, but the middle one was a Scanner. The Strategist wrote search patterns, the Scanner ran them as regular expressions over the files that imported the library, and the Auditor received thirty lines above and below every match.\n\nNoise went down. Then it met this:\n\n```\nconfig = yaml.safe_load(open(\"settings.yml\"))   # safe\ndata   = yaml.load(request.body)                # vulnerable\n```\n\nA text search finds both lines. The Auditor gets two blocks of code with no idea where `request.body` comes from or who calls the surrounding function. It has to guess, and on a bad day it guesses `not_affected`.\n\nI replaced the Scanner with a Locator that queries a code graph. The Scanner stayed in the codebase with two smaller jobs, covered below. The division of labour stayed the same, and the rest of the post builds on it: the LLM reads the CVE at the start and judges facts at the end. The graph does the looking.\n\nA CVE is written for people. \"Heap overflow in the TIFF decoder when processing crafted images\" does not tell you which function in your code triggers it, and no database does either. NVD used to add structured data such as scores and affected configurations to every entry; in April NIST announced it would stop enriching every CVE. Reading an advisory and mapping it to code used to need someone with security experience, every time.\n\nThe Strategist does that reading. It receives the CVE description and the package name and returns a plan as strict JSON. For CVE-2017-18342 (PyYAML before 5.1, `yaml.load()` can execute arbitrary code on untrusted data) the plan looks like this, with illustrative values:\n\n```\n{\n  \"trigger_type\": \"api_call\",\n  \"vulnerable_symbols\": [\"yaml.load\", \"yaml.load_all\"],\n  \"conditions\": \"the argument comes from untrusted input and no SafeLoader is passed\",\n  \"safe_alternatives\": [\"yaml.safe_load\", \"yaml.safe_load_all\"],\n  \"import_aliases\": [\"yaml\"],\n  \"attack_vector\": \"a YAML document with a python/object tag executes code when loaded\"\n}\n```\n\nEach field has a consumer downstream. The Locator uses `vulnerable_symbols` and `import_aliases` (Pillow is imported as `PIL`, PyYAML as `yaml`). The Auditor uses `conditions`, `attack_vector` and `safe_alternatives`.\n\nThis is the first place a real vulnerability can disappear. If the Strategist names the wrong function, the graph looks for the wrong thing. This part of the prompt guards against it:\n\n```\n- vulnerable_symbols: ALL the PUBLIC functions/classes an application may call that\n  lead to the vulnerable code, fully qualified from the TOP-LEVEL import name.\n  If the bug is inside an internal decoder or C module (e.g. TiffDecode.c), name\n  the public entry points that reach it (e.g. \"Image.open\" for image decoders).\n  If it is in a class used internally by convenience functions, list those too\n  (e.g. a bug in requests.Session redirects: \"requests.get\", \"requests.post\", ...).\n  Missing an entry point hides a real vulnerability, so when unsure list more, not fewer.\n```\n\nIn the Pillow example the bug lives in a C decoder that no application calls directly. Applications call `Image.open`, which reaches the decoder when it opens a TIFF. The prompt asks the Strategist to make that jump, from where the bug is to where your code touches it.\n\nThe plan also says how the bug fires:\n\n`api_call`: the application must call a specific function.` framework_implicit`: the library acts on its own. A CVE in Django's multipart parser fires on every upload before your view runs, and your code never names the parser.`config`: a setting turns the bug on.\nThe Strategist picks the type from the CVE text, and the type decides which question the Locator asks.\n\nNothing uses the answer before validation. The JSON must parse, the trigger type must be one of the three values, and symbol names are normalised. If Bedrock is unreachable, a keyword heuristic produces a fallback plan and the pipeline marks it as heuristic, because a guessed plan must never clear a CVE. Plans are cached per run, so each CVE/package pair goes to the model once.\n\nA wrong symbol list is the cheapest way to hide a vulnerability. Ask for more entry points, never fewer, and record where each plan came from.\n\nThe Locator uses no model. It reads the graph that [OWASP dep-scan](https://github.com/owasp-dep-scan/dep-scan) builds with atom, which combines three views of the same code. The syntax tree knows that `Image` in your file is Pillow's module. The call graph knows who calls whom. The data flow knows where a value travels.\n\ndep-scan exports the graph as two JSON slices. `usages` lists every call with its arguments and line. `reachables` lists flows that start from external input, such as an HTTP request. The CI job pins the image, because the slice format changes between atom versions, and writes a marker file only when the slices are complete:\n\n```\nvex_reachability:\n  stage: analyze\n  image:\n    name: ghcr.io/owasp-dep-scan/dep-scan:v6.3.0\n    entrypoint: [\"\"]\n  allow_failure: true\n  script:\n    - |\n      if depscan -i \"${VEX_APP_SRC}\" -o \"${CI_PROJECT_DIR}/depscan-out\" -t python \\\n        --reachability-analyzer SemanticReachability \\\n        --explanation-mode EndpointsAndReachables \\\n        --vdb-scope app --no-banner \\\n        && [ -s depscan-out/python-usages.slices.json ]; then\n        touch depscan-out/.complete\n      fi\n```\n\ndep-scan exits 0 even when it produces no slices. Partial slices would report \"library not used\" for every file that failed to parse, and the analysis would close CVEs it never looked at. Without `.complete`, the Locator treats the graph as missing.\n\nMatching is one small function. atom writes resolved calls such as `PIL.py:<module>.Image.open`. The Locator splits them into segments and compares from the end:\n\n``` php\ndef symbol_matches(qualified: List[str], symbol: str) -> bool:\n    sym = [s for s in _norm(symbol).replace(\"()\", \"\").split(\".\") if s]\n    if not sym or len(sym) > len(qualified):\n        return False\n    return qualified[-len(sym):] == sym\n```\n\n`PIL.Image.open` matches `[pil, image, open]`. `yaml.load` matches `yaml.py:<module>.load` and never `safe_load`, because the last segment differs. That fixes the case that sank the Scanner.\n\nPin the dep-scan image and check that the slices exist. A graph that failed silently looks exactly like a codebase that never uses the library.\n\nThe graph can give four answers, and each has a fixed next step.\n\nFor every call site the Locator collects the file and line, the origin of each argument, the external-input flows that cross the enclosing function, every path from an entry point to the call, and the source of each function on those paths. All of it goes to the Auditor. This is the Pillow case: `Image.open` is called at line 17 of `upload_avatar`.\n\nThis looks like good news, and it is the outcome most likely to hide a real problem, because the Strategist can still miss an entry point. Take a CVE in the redirect handling of `requests.Session`. The Strategist names `requests.Session`. The application only does this:\n\n```\nr = requests.get(user_supplied_url)\n```\n\n`requests.get` builds a `Session` internally, so the vulnerable code runs. Stopping at \"not called\" would be wrong. Before trusting it, the Locator runs three checks, one for each way \"not called\" could be false:\n\n| Check | Fails when | Why it matters | \n|---|---|---|\n| The symbols came from the LLM | Bedrock was down and the keyword heuristic produced them | a guessed list proves nothing | \n| They are real, callable APIs | the plan names `TiffDecode.c` , a private`_imaging` module or a decoder plugin | no application calls a C file, so \"not called\" is trivially true | \n| A text search agrees | grep finds `ImageMath.eval(` where the graph saw nothing | the graph has a hole: an unparsed file, an alias, a `getattr` | \n\nAny failure sends the CVE to `in_triage`. If all three pass, the Auditor receives every use of the library and one question: does any of these calls reach the vulnerable code internally?\n\nFor `framework_implicit` and `config` there is no symbol to look for. If the library is imported, the Locator collects every place the application uses it and the Auditor decides whether the vulnerable feature is in use. On the lab app it sees `upload_avatar` reading `request.FILES`, so multipart uploads are in use.\n\nIf the library is not imported anywhere, the CVE goes straight to `in_triage`. A server, a plugin or a string in the settings can load a package without any import in your code.\n\nIf dep-scan failed or the language is not Python, the old Scanner runs. If it finds uses, the Auditor judges. If it finds nothing, the CVE goes to `in_triage`, because without the graph \"not found in the text\" proves nothing.\n\nThe relationship between the graph and the regex comes down to one rule:\n\nIf the graph says \"called\", believe it. If it says \"not called\" or \"not imported\", the text has to agree. When they disagree, doubt wins.\n\nFor an `api_call` CVE, \"never imported\" sounds conclusive. It isn't, because the graph only sees your code. If Django uses sqlparse internally, a sqlparse CVE can be reached through Django, and the graph never looks inside Django.\n\nAn SBOM generated from `requirements.txt` has no dependency tree. The Locator rebuilds one from the `requires_dist` metadata on PyPI, breadth-first from the SBOM's components. On the lab app that is 34 packages in about two seconds.\n\nPyPI cannot tell you whether Django calls the vulnerable function in sqlparse. It only tells you that Django declares a dependency on it. That is enough to know when a CVE must stay open:\n\n| Library | PyPI tree | Verdict | \n|---|---|---|\n| jinja2 | direct, nothing else requires it | `not_affected` if no text search finds an import | \n| sqlparse | required by Django | `in_triage` | \n| private package | PyPI returns 404 | `in_triage` | \n\nThe jinja2 row is the only case the Locator closes on its own, without calling the LLM. Before it does, a regex looks for any import of the package, including dynamic ones through `importlib` or `__import__`.\n\nOptional extras count as dependents, but the walk does not traverse them. Following every extra makes every library transitive (Pillow → `docs` extra → sphinx → jinja2) and nothing could ever be closed. The known trade-off: a package reached only through another package's extra, such as `requests[security]` → pyOpenSSL → cryptography, can look direct.\n\nThe graph sees your code. PyPI sees what your dependencies declare. Neither one is allowed to clear a CVE on a guess.\n\nThe Auditor receives the Strategist's plan, the CVE description and a block of evidence built from the slices. The prompt tells it what kind of material it is reading:\n\n```\nEVIDENCE FROM THE CODE GRAPH (OWASP dep-scan). For every call site of the\nvulnerable API you get: the arguments' origin, the external-input flows that\ncross the enclosing function, every call path from an entrypoint, and the code\nof every function on those paths. These are facts from data-flow analysis, not\nguesses: the vulnerable API IS called at these lines.\n\nFor each call site decide:\n1. Can attacker-controlled data (HTTP request, uploaded file, message, CLI arg)\n   reach the call through one of the paths? Follow the arguments' origin.\n2. Are the trigger conditions met (arguments, options, file format, settings)?\n3. Is there validation or a safe alternative that blocks the attack?\nThe CVE is \"affected\" if at least one call site is exploitable.\n```\n\nA rule engine can't answer these questions. Is there validation before the call? Is the argument a constant or an upload? Does the file format match the CVE? Is `yaml.load` called with a `SafeLoader`? A person answers in seconds when looking at the right snippet. The graph's job is to put that snippet, and the path that leads to it, in front of the model.\n\nThe evidence is capped so prompts stay small: at most 8 call sites per CVE, 5 paths per call site, call depth 8, and 80 lines per function. That is enough to follow a request from a Django view to a library call in a typical web app.\n\nThe Auditor answers with strict JSON: a verdict that is exactly `affected` or `not_affected`, a confidence between 0 and 1, a reasoning that cites the path, and the call sites it considers exploitable.\n\nThe graph decides what the model sees. The model decides what it means.\n\nIn case 2 above (library used, vulnerable function not called) the Auditor is asked whether `requests.get` reaches `Session` internally. Nothing in the evidence answers that. The engine passes no library source, no documentation and no web search, so the model answers from training data.\n\nFor widely used libraries such as requests, Pillow, PyYAML and Django, that is usually right: the mapping between public functions and internal classes is stable and well documented. For a niche package, or a release newer than the model's training cutoff, it can be wrong. A smaller model knows less than a larger one.\n\nTwo things contain this. The confidence threshold sends an unsure answer to `in_triage`, which is what usually happens with a library the model barely knows. And every reasoning is written into the VEX and committed, where a reviewer can read and challenge it. A model that is confidently wrong passes the threshold; only the review catches it.\n\nThe next step is to replace memory with facts: have atom also build a graph of the installed library and show the Auditor the real path from `requests.get` to `Session`. vex-engine does not do this yet.\n\nThe final rule does not depend on the model:\n\n``` php\n# Fail-closed: None (doubt, error, low confidence) -> in_triage, stays visible\nif is_reachable is None:\n    state = \"in_triage\"\nelse:\n    state = \"exploitable\" if is_reachable else \"not_affected\"\n```\n\nA verdict with confidence of 0.8 or more (`VEX_MIN_CONFIDENCE`) becomes `exploitable` or `not_affected`. Everything else becomes `in_triage` and stays on the dashboard for a person: a malformed answer, a lower confidence, a timeout, a missing package URL from Dependency-Track.\n\nOne failure mode only shows up over time. An LLM is not deterministic, and on an ambiguous CVE it can flip between `not_affected` and `exploitable` on every run, closing and reopening the same issue. The bot counts its own flips. After two (`VEX_MAX_AUTO_FLIPS`) it stops touching the issue, leaves it open and labels it `vex::needs-human`. A human decision resets the counter.\n\nThis rule is what makes a smaller model safe to use. I ran the whole pipeline with a 12B open model through Ollama on a laptop. It produced more `in_triage` verdicts than the Bedrock model and no hidden errors.\n\nWorst case, vex-engine leaves you with the dashboard you already had.\n\nBack to the Pillow decoder CVE on the lab app (old Pillow, PyYAML and Django, with insecure code written on purpose).\n\nThe Strategist returns `trigger_type: api_call` with `PIL.Image.open` among the symbols, because that is the public function that reaches the decoder. The plan came from the model, and the symbol is a real API.\n\nThe Locator finds `Image.open` in the graph and builds the evidence. Simplified:\n\n```\nCall site 1: Image.open @ vuln_demo.py:17 (in upload_avatar)\nVulnerable symbol: PIL.Image.open\nArguments: img_file <- request.FILES['avatar']\nCall paths (entrypoint -> call):\n  <module> --(line 56)--> upload_avatar --(line 17)--> Image.open\nCode of the functions involved:\n  def upload_avatar(request):\n      ...\n      img_file = request.FILES['avatar']\n      with Image.open(img_file) as img:\n```\n\nThe Auditor does not have to find the call or trace the data. It only decides whether an attacker controls that file: it is a user upload, and nothing validates it before the decoder runs. Verdict `affected`, confidence above the threshold, call site `vuln_demo.py:17`.\n\nThe result is a VEX entry marked `exploitable` and a GitLab issue for the Pillow component with the file, the line, the origin of the argument and the path from the entry point. The developer who opens it can go straight to the fix.\n\nA full run on the lab app looks like this. The app is vulnerable by design, so the exploitable share is far higher than in a real codebase:\n\nOf 80 CVEs, 40 are closed with a written reason, 6 stay visible for a person, and 34 exploitable ones are grouped into two issues, one for Pillow and one for Django.\n\nEvery verdict is written to `vex.json` in the application's repository, using standard CycloneDX fields, and committed with `[skip ci]` only when something changed:\n\n```\n{\n  \"id\": \"CVE-XXXX-YYYY\",\n  \"analysis\": {\n    \"state\": \"not_affected\",\n    \"justification\": \"code_not_reachable\",\n    \"response\": [\"will_not_fix\"],\n    \"detail\": \"Vulnerability CVE-XXXX-YYYY analyzed by LLM Analyzer (python) on 2026-10-20. <Auditor reasoning>\"\n  }\n}\n```\n\nIn the uploaded VEX, `affects[].ref` must stay the Dependency-Track project UUID (the BOM's `metadata.component.bom-ref`). If you rewrite it with the component's purl, Dependency-Track returns HTTP 200 and silently discards the verdict. The only trace is `Unable to locate affected element` in its logs. vex-engine keeps the purl in a separate property, `vex:component-purl`.\n\nIssues open only for exploitable CVEs, one per component. People close them with a label that says why, and the pipeline turns the label into a VEX decision:\n\n| Label | VEX written | Expiry | What the bot does | \n|---|---|---|---|\n| `vex::fixed` | nothing forced, it's a claim | none | next analysis checks the graph; still reachable → issue reopens | \n| `vex::false-positive` | `false_positive` | 90 days | skipped until expiry, then back to `in_triage` | \n| `vex::mitigated` | `not_affected` +`protected_by_mitigating_control` | 90 days | same | \n| `vex::risk-accepted` | `exploitable` +`will_not_fix` | 90 days | stays visibly exploitable | \n\nIf `VEX_APPROVERS` is set and someone else closes the issue, the bot reopens it. Two `vex::` labels at once count as a conflict. A person reopening an issue revokes the decision. When a library is upgraded and its CVEs leave the SBOM, the issue closes by itself.\n\nA decision that never expires becomes a blind spot. Ninety days is a setting. The expiry itself is the point.\n\nA pipeline that runs only on push misses new CVEs on code nobody touches, and misses the decisions people make on issues. vex-engine runs on three triggers:\n\n`VEX_MODE=critical`).` vex::` label to a closed one, runs `vex_issue_sync --issues-only`: no scan, no LLM, Dependency-Track updated in about a minute.`VEX_MODE=full`) catches new CVEs on unchanged code.\nTwo jobs write `vex.json`, so they share a GitLab resource group and run one at a time. Set its process mode to `oldest_first` to keep them in order. A project enables everything by including one template:\n\n```\ninclude:\n  - project: 'demo-security/vex-engine'\n    file: 'ci-template.yml'\n    ref: main\n\nstages: [scan, upload-bom, analyze, sync-vex]\n```\n\nThe LLM is called in two places, the Strategist and the Auditor, and only during an analysis. The SBOM, Dependency-Track, the graph, the webhook and the nightly sync make no model calls. The Strategist runs once per CVE in scope; the Auditor runs only for CVEs the graph does not settle on its own.\n\nThe default model is Amazon Nova 2 Lite. The task is narrow and structured (read text, return JSON), so a small model is enough. Switching model is one variable, and the same prompts run locally through Ollama, in which case no code leaves the machine:\n\n```\nLLM_PROVIDER=bedrock                         # or: ollama\nBEDROCK_MODEL_ID=eu.amazon.nova-2-lite-v1:0  # default\nOLLAMA_MODEL=gemma4:12b                      # local option\nVEX_MIN_CONFIDENCE=0.8\n```\n\nA run costs cents on Bedrock and nothing locally. The real infrastructure cost is the graph: atom runs on the JVM and the dep-scan image is about 3.8 GB, so cache it on your runners.\n\nEach of these ends in `in_triage`, never in `not_affected`:\n\n| Component | Configuration | What it guarantees | \n|---|---|---|\n| Strategist prompt | all public entry points, more when unsure | the graph looks for every way in | \n| Fallback plans | marked as heuristic | a guessed plan never clears a CVE | \n| dep-scan job | pinned image, `.complete` marker | a failed graph is treated as missing | \n| Symbol matching | segment suffix match | `yaml.load` never matches`safe_load` | \n| \"Not called\" checks | LLM origin, real API, regex agreement | a hole in the graph ends in triage | \n| PyPI tree | direct leaf only, extras not traversed | transitive libraries stay visible | \n| Auditor evidence | 8 sites, 5 paths, depth 8, 80 lines | small prompts with the right snippet | \n| Verdict rule | `VEX_MIN_CONFIDENCE=0.8` , fail closed | doubt never hides a CVE | \n| Anti-oscillation | `VEX_MAX_AUTO_FLIPS=2` ,`vex::needs-human` | an unstable verdict goes to a person | \n| VEX upload | keep `affects[].ref` as the project UUID | Dependency-Track keeps the verdicts | \n| Labels | approvers, 90-day expiry | decisions are attributed and revisited | \n| Writers | shared resource group | concurrent jobs don't drop decisions | \n\nEverything here ran with OWASP dep-scan 6.3.0, Dependency-Track 4.13 and Amazon Nova 2 Lite on Bedrock, on Python projects. The slice format changes between atom versions, so expect the matching code to move.\n\nTo run it on your own project: clone [vex-engine](https://gitlab.com/argomauro/vex-engine), point `VEX_APP_SRC` at your source, add the `include:` block above to your `.gitlab-ci.yml`, and start with `LLM_PROVIDER=ollama` if you'd rather keep code on your machine for the first run. The lab app in the repo is the fastest way to see all four routing cases.", "url": "https://wpnews.pro/news/cve-reachability-with-an-llm-and-a-code-graph", "canonical_source": "https://dev.to/margoneto/cve-reachability-with-an-llm-and-a-code-graph-cl0", "published_at": "2026-10-30 13:10:12+00:00", "updated_at": "2026-10-30 13:21:17.774609+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "developer-tools", "ai-safety", "mlops"], "entities": ["vex-engine", "Amazon Bedrock", "OWASP dep-scan", "GitLab", "Dependency-Track", "Trivy", "NVD", "NIST"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cve-reachability-with-an-llm-and-a-code-graph", "markdown": "https://wpnews.pro/news/cve-reachability-with-an-llm-and-a-code-graph.md", "text": "https://wpnews.pro/news/cve-reachability-with-an-llm-and-a-code-graph.txt", "jsonld": "https://wpnews.pro/news/cve-reachability-with-an-llm-and-a-code-graph.jsonld"}}