Where the LLM Stops: Deterministic Scoring in an AI-Assisted VAPT Pipeline ONUS, an open-source, self-hosted DAST platform developed at IIT Kanpur, ensures that all numeric scores in its vulnerability reports come from deterministic formulas rather than its local language model. The project's architecture, validated by 690 tests, guarantees that re-running scans produces byte-identical numbers even when the LLM is slow or unreachable, degrading only the prose. This design choice prioritizes reproducibility and availability over the variability of model outputs. Every VAPT report ends the same way: a handful of numbers. A CVSS score. A severity label. A priority rank. Sometimes an aggregate risk score. Those are the numbers a remediation team actually acts on: what gets patched this sprint, and what waits. Once a large language model enters that pipeline writing summaries, explaining findings, drafting remediation steps , a quieter architectural question follows it in: is the model narrating scores that already exist, or is it, somewhere along the way, actually shaping them? This is a technical write-up of how ONUS, an open-source, self-hosted DAST dynamic application security testing platform, answers that question by construction rather than by policy. It's drawn from ONUS's internal project report, its architecture, its test suite, and scan data collected during development and validation, with the test count independently verified against the live repository more on that below , not from GitHub stars, scan counts, or its listing in OWASP's Vulnerability Scanning Tools directory. None of those are evidence that an architecture is sound, and this article deliberately sets them aside in favor of what was actually built, tested, and observed. Links to the repository, project site, and that listing are at the end, as references, not as proof of anything. ONUS's own validation confirms the design held in practice: every scored field traces to a single function, so re-running a scan against an unchanged target reproduces identical numbers. When the local language model is slow or unreachable, only the report's prose degrades. The severity, CVSS, and priority numbers never move. I originally built and validated ONUS as a supervised project for the IIT Kanpur Computer Centre, under Navpreet Singh, before open-sourcing it. For readers meeting it for the first time: ONUS is a self-hosted DAST platform. An operator submits an authorized domain; eight scanning modules run against it in parallel; results are deduplicated, passively re-verified, scored against the official CVSS v3.1 formula the industry-standard 0–10 vulnerability severity scale , described in plain English by a local language model, and delivered as both a downloadable PDF and a live dashboard. Three commitments run through the whole system: That architecture isn't really this article's subject, though. What is: the line ONUS draws around what its LLM is allowed to touch, and what happens on both sides of that line when things go wrong. ONUS's project report states this as one of its core design objectives, in plain terms: every numeric score in the final report CVSS score, CVSS vector, severity, priority, aggregate risk score has to come from a deterministic formula, never from the language model, so that running the same scan twice produces byte-identical numbers. That's a strong claim. It's worth asking why it's the right one, not just noting that ONUS makes it. Reproducibility is testable; "usually right" isn't. A pure function cvss scorer.py::score finding can be checked against known CVSS vectors in a unit test and asserted to never drift. ONUS's test suite 690 tests, as of this writing does exactly that: the CVSS formula is validated against known vectors as one specific, ongoing test category. There's no equivalent test you can write for "the model rates this finding as Medium," because a model's output can vary across prompts, model versions, and for a self-hosted, comparatively small model like the 7B one ONUS runs to stay air-gapped whether it's even reachable that day. Availability and correctness shouldn't share a failure mode. If the LLM produced the severity number, LLM downtime would force a choice: block the report entirely, or quietly fall back to some second, less-tested scoring path: exactly where a less-tested path is most dangerous. ONUS's own discussion of results confirms this decoupling held in practice: a slow or unreachable Ollama instance degrades the report's prose quality, but never its severity, CVSS, or priority numbers. When Ollama can't be reached after retries, a rule-based fallback description is substituted and the report is explicitly flagged ai unavailable , never silently left blank, and never routed through a different scoring path. The model never sees evidence before it's scored. This is structural, not just policy: aggregation, passive re-verification, and CVSS scoring all happen, and all complete, before the LLM is invoked at all. By the time Qwen 2.5 7B sees a finding, it's already deduplicated, already tiered by confidence, and already carries its final CVSS score and priority. Its only job from there is to write about it. That last point has a second, more general benefit worth naming, even though it isn't a claim the report itself makes or set out to test: a scanner's raw evidence often includes text pulled directly from the target: page titles, error strings, response bodies. That's attacker-adjacent content, and occasionally attacker-controlled. A model that reads raw target content and assigns the severity number is a model whose score a sufficiently motivated target could, in principle, try to influence. A model that only writes prose about a number it has no ability to change has a much smaller blast radius if something in that evidence turns out to be adversarial. Scoring first and describing second closes that door structurally, whether or not it was the original motivation for the design. ONUS is a six-layer pipeline. Each layer has one job and talks to its neighbors. | Layer | Component | Responsibility | |---|---|---| | 1: Input | Next.js frontend | Domain entry form, authorization checkbox, live scan status | | 2: Backend | FastAPI + PostgreSQL | Request validation, job creation, status API, report delivery | | 3: Queue | Celery + Redis | Async dispatch, parallel worker orchestration, task state | | 4: Scanning | 8 Python modules | Execute external tools, normalize output to a shared JSON schema | | 5: Intelligence | Ollama + Qwen 2.5 7B | CVSS scoring, risk ranking, remediation prose | | 6: Output | WeasyPrint + Next.js | PDF report, interactive vulnerability dashboard | That Layer 5 label is a simplification: the architecture diagram it's drawn from only has room for one box. Layer 5 is actually four sequential stages, and only the last one touches Ollama at all. That's the subject of "Inside the Analysis Pipeline," below. A scan starts when an operator submits a domain with authorization confirmed. FastAPI validates the request, rejecting private IP ranges RFC 1918 and localhost outright, checks for duplicate or concurrent scans against the same domain, creates a Scan row, and pushes a job to Redis. Celery dispatches a group of eight parallel scanning subtasks; a chord callback a Celery primitive that fires once every task in a group has completed triggers once all eight report back, and only then does aggregation, verification, scoring, description, and PDF rendering begin. Two schema-level choices are worth calling out, because they show the layering is enforced in the database, not just the diagram. The generated PDF is stored in a separate reports table as BYTEA , deliberately kept apart from the scans row, so that polling a scan's status never has to read or write binary PDF data. And updates to a scan's per-module status map use a raw, atomic jsonb set SQL statement rather than a read-modify-write ORM update, specifically to avoid a race condition when multiple parallel Celery workers try to update the same scan's status at once. WeasyPrint, notably, is also what rendered ONUS's own underlying project report: same renderer, both jobs. | | Module | Tools | Finds | |---|---|---|---| | 1 | Recon | nmap, subfinder, Amass, httpx, Naabu, WHOIS, dnspython | Ports/services, subdomains, live-host tech, WHOIS, DNS/SPF/DMARC/DKIM | | 2 | Web Scan | OWASP ZAP, Nikto, Katana | XSS/SQLi/CSRF/broken auth, misconfigurations, JS-aware endpoints | | 3 | SSL/TLS | testssl.sh, sslscan | Protocol/cipher issues, certificate validity, HSTS | | 4 | Headers | pure requests | CSP/HSTS/X-Frame-Options/CORS/cookie flags | | 5 | OWASP Top 10 | requests , 6 test functions | SQLi, XSS, IDOR, path traversal, open redirect, error disclosure | | 6 | Tech Fingerprint | WhatWeb, WAFW00F | CMS/framework/server detection, WAF Web Application Firewall presence | | 7 | Nuclei CVE | Nuclei | Known CVEs, misconfigurations, exposed panels | | 8 | Dir Enum | FFUF | Exposed files, admin panels, auth-gated paths | None of these tools is novel: nmap, ZAP, Nikto, testssl.sh, and Nuclei already do their individual jobs well on their own. What none of them do alone, and what the project report frames as the actual contribution over just running them separately, is operate in one coordinated pipeline against a single target, deduplicate and cross-reference their overlapping findings, apply one consistent formula-derived score across all of them, and produce a single narrative a non-technical reader can act on. Every module wraps its external tool via subprocess under a controlled timeout, and every module has to normalize its output to one shared finding schema: module, tool, type, title, evidence, target, found by , and a confidence/verifiable flag. That schema is treated as non-negotiable throughout the codebase, for a specific reason: a module that emits a malformed finding causes silent data loss at the aggregation stage. Hold onto that: it comes back later, once as a design decision and once as a real bug. For targets that sit behind authentication, an operator can supply login credentials on submission. ONUS stores them in Redis, keyed by scan ID, never as a Celery task argument, and never written to the scans table. A scanning module retrieves the credentials, auto-detects whether the login is an HTML form or a JSON API, logs in, and crawls and tests only the authenticated surface. Logout-shaped links are explicitly excluded from the crawl: a rule that exists because of a real bug, covered below. Credentials are deleted from Redis once the scan finalizes. Once all eight modules report back, ONUS runs a fixed four-stage pipeline before anything reaches a human, or an AI model acting as narrator. This pipeline and the fact that its stages run in a fixed order, each depending on the last one having already finished is the actual trust boundary this article is about. php flowchart TD A "8 module result envelopes" -- B "Aggregator