{"slug": "i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me", "title": "I built a vulnerability scanner that refuses to lie to me", "summary": "A developer built secfix, a vulnerability scanner that validates findings by executing code in a sandbox rather than relying on syntax analysis or AI-generated descriptions. The tool confirmed a real SQL injection in a Django view by running the code, but required manual input for framework-specific details, revealing the boundary of what execution-based vulnerability reproduction can achieve autonomously.", "body_md": "Static analysis tools have a trust problem. They read your source, pattern-match something that looks injectable, and hand you a warning — plus, increasingly, an AI-generated \"fix.\" You have no idea if the warning is real or if the fix works. You just get more things to check.\n\nI wanted the opposite: a tool that doesn't guess, doesn't trust the AI's word, and doesn't call anything fixed until it's proven fixed. So I built secfix.\n\nThe idea: proof by execution\n\nGiven a finding from Semgrep, secfix doesn't re-read the code. It runs it. It builds a test harness that calls the flagged function with a unique tainted marker, executes it inside a locked-down Docker sandbox, and records exactly where that marker ends up. If the marker lands unescaped inside a SQL string, or in a shell command, or in a file path that escapes its directory — that's a confirmed vulnerability, demonstrated by execution, not inferred from syntax.\n\nThen it patches the function, runs the same harness again on a fresh trace, and only calls the fix validated if the marker now lands somewhere safe. Crucially: it never trusts the model's own description of what it changed. The AI's account of the fix is irrelevant — only a fresh execution trace counts. If re-verification doesn't come back clean, the patch is reported as unvalidated, full stop.\n\nThe whole thing is built to fail closed. When it can't prove something, it says uncertain — it never rounds up to \"safe.\"\n\nWhere it broke — and why that's the interesting part\n\nThe tool passed all its tests. But passing your own fixtures proves very little. So I pointed it at real vulnerable Django apps.\n\nIt hit a wall immediately. Framework code isn't like a plain function — you can't just import a Django view and call it. The moment you touch it, Django wants settings, an app registry, a database, a request object. My harness imported the module and crashed before it reached a single line of the actual vulnerability.\n\nI could have documented that as a limitation and moved on. Instead I went after it, one wall at a time: detecting the framework and its required Python version, matching the sandbox base image to it, calling django.setup(), baking a migrated database into the container, and constructing a real HTTP request with Django's own RequestFactory.\n\nAnd it worked — I got a real Django view SQL-injection all the way to a confirmed verdict, the tainted input landing unparameterized in executed SQL, proven by running it.\n\nThe honest conclusion\n\nBut reaching that verdict took something I'm not going to pretend away: I had to hand-feed the tool knowledge it couldn't derive on its own — the specific database row the view expected, and a framework-internal file-size threshold buried in Django's upload handling. Some walls generalized cheaply. Others didn't — and the ones that didn't are exactly the ones that separate a clever solo project from the multi-year integration problem that funded security companies work on with whole teams.\n\nI wrote all of it down — which walls fell easily, which didn't, and precisely where \"automatic\" stops. That map is the real deliverable. Not a demo where everything works, but an honest boundary of what execution-based vulnerability reproduction can and can't do on its own today.\n\nThe code, the safety design, and the full investigation are here: [https://www.github.com/balbaks/secfix](https://www.github.com/balbaks/secfix) — with the Django deep-dive on the v1-django-bootstrap-spike branch.", "url": "https://wpnews.pro/news/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me", "canonical_source": "https://dev.to/balbaks/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me-22fl", "published_at": "2026-08-24 17:35:51+00:00", "updated_at": "2026-08-24 18:14:02.120617+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-safety"], "entities": ["Semgrep", "Django", "secfix", "GitHub", "balbaks"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me", "markdown": "https://wpnews.pro/news/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me.md", "text": "https://wpnews.pro/news/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me.txt", "jsonld": "https://wpnews.pro/news/i-built-a-vulnerability-scanner-that-refuses-to-lie-to-me.jsonld"}}