SecurityArticle Cheap LLM bug-hunting has collapsed the signal-to-noise of disclosures, and the old etiquette of coordinated reporting no longer pays for itself.
[Ji-ho Choi](https://www.devclubhouse.com/u/jiho_choi)
For most of the last two decades, a vulnerability report carried a special status that an ordinary bug report never did. Open a feature request and a maintainer owes you nothing. Send something to `security@`
and the unwritten contract kicks in: prompt acknowledgement, an investigation, status updates, and eventually a credit line. Filippo Valsorda, former lead of the Go security team, spent years drilling that distinction into new hires. His recent argument is that the contract has quietly expired, and he's largely right, though for reasons worth pulling apart more carefully than a single throwaway line about running LLMs in CI suggests.
The claim is uncomfortable precisely because the etiquette around disclosure was hard-won. It existed for a real economic reason, and that reason has changed.
Why reports were ever "special" #
The deal was never about the researcher. It was about two scarce goods they controlled: insight and confidentiality. Finding a pre-auth RCE in widely deployed software took skill that few people had and fewer would spend on your project for free. And a researcher who found one could either hand it to you privately, giving you a window to ship a fix before exploitation, or drop it publicly and let the race begin. Responsiveness and attribution were the price maintainers paid for that window.
There's history baked into this. In an earlier era, reporting a bug could get you a lawsuit or a prosecution, and the full-disclosure movement existed partly to shame the industry out of that. Coordinated disclosure was a truce: researchers held fire, vendors stopped threatening them. That truce still matters in some corners, but as Valsorda notes, it's mostly irrelevant to open source in 2026. No serious project threatens reporters anymore.
What actually broke is the scarcity. When an LLM can scan a codebase and surface plausible-looking issues in minutes, insight stops being rare. The maintainer can run the same model the attacker runs. The bottleneck moves from finding candidate issues to assessing which ones are real, and that's a job an external reporter usually can't help with unless you already trust them.
The cost moved to triage, and the noise is brutal #
Any maintainer who runs a public security@
inbox will recognize the shape of this. The volume of low-quality, model-generated reports has climbed sharply, and picking through them has roughly the same signal-to-noise ratio as reading raw LLM output. You get confident prose, fabricated call paths, CVE-shaped text that doesn't reproduce, and the occasional genuine finding buried underneath. The work isn't reading the report. It's disproving it.
That said, the "LLMs are as good as almost any security researcher" framing oversells the present. Look at what's actually getting disclosed. Squidbleed (CVE-2026-47729), a heap over-read in Squid that leaks another client's cleartext HTTP request, traces back to a 1997 FTP-parsing change and was found by researchers at Calif.io. DifyTap, four cross-tenant flaws in Dify reported by Zafran Security, required understanding a multi-tenant plugin daemon well enough to traverse it unauthenticated. These are not slop. They're the kind of deep, context-heavy work that still rewards a skilled human.
So the reality is two-tier. The floor has dropped out, flooding inboxes with cheap noise, while the ceiling, real novel research, still comes from people who know what they're doing. The mistake would be to let the noise make you cynical about the signal. The job is to build a triage process that separates them fast, not to stop reading.
What development teams should actually change #
The practical fallout lands on anyone who maintains software others depend on. A few concrete shifts:
Gate triage behind reproduction, not politeness. The old contract promised responsiveness to the reporter. Reframe it around the report. A finding that ships with a working proof of concept, a failing test, or a reproducible trace gets your time. One that's a wall of model-generated speculation gets a templated request for repro and nothing more until it arrives. This isn't rude. It's the only way to keep a volunteer-run inbox functional when the cost of generating a plausible report has gone to nearly zero.
Trust relationships become the filter. When insight isn't scarce, provenance is. A report from someone with a track record, or routed through a known disclosure program, is worth triaging first. Anonymous mass-submitted output goes to the back of the queue. This is uncomfortable for newcomers trying to break in, and it's a real cost, but it's where the incentives now point.
Prioritize by exploitability, not CVSS. The score lies in both directions. Joomla's CVE-2023-23752 carries a mild CVSS of 5.3, yet Shadowserver flags it as actively swept because exploitation is public, trivial, and many vulnerable endpoints leak plaintext passwords. Gravity SMTP's CVE-2026-4020 is another 5.3 information-disclosure bug that attackers used to pull API keys and OAuth tokens off roughly 100,000 WordPress sites. A medium score on a thing with a public one-liner exploit outranks a 9.8 that needs a lab to trigger. Triage on "is this being used" before "what's the number."
Treat "run LLM analysis in CI" as a real engineering task, not a footnote. The symmetry that floods your inbox also raises your floor: the same models can pre-screen your own code on every pull request. But this is not free. You'll fight false positives, you'll burn tokens, and you'll need a human gate before anything reaches a triage queue. Done well it shifts findings left, into the diff, before they ever become a security@
email. Done badly it just builds your own slop generator pointed at your own team.
The embargo math, and where it still holds #
Valsorda's sharpest and most contestable point is that confidentiality, embargoes, and coordination matter far less than they did, because the attacker doesn't need your disclosure post. They can ask their own model, and they probably have the same triage bottleneck you do. There's something to this. The defensive value of an embargo was always the head start it bought, and that head start shrinks when both sides can independently rediscover the same bug.
I'd hold this one more loosely than the rest. Embargoes still buy coordination across a supply chain, time for downstream distributors to stage patches, for cloud providers to deploy mitigations, for the long tail of vendors who ship the same vulnerable library. The Shadowserver data is the argument for why that window still matters: years after patches existed for Fortinet, Citrix, VMware ESXi and others, the internet is full of unpatched, exploited instances. FortiBleed alone identified working credentials on more than 80,000 FortiGate devices. The problem there was never disclosure timing. It was patch velocity. If anything, weakening the embargo norm raises the premium on shipping and deploying fixes fast, because the head start you're losing is exactly the one those laggards depend on.
The honest read is that the etiquette is dying but the responsibility isn't. Maintainers never owed researchers anything intrinsically. They owed their users safety, and the report was the mechanism. That mechanism is now noisy and cheap, so the work moves to where the leverage is: ruthless triage, fast remediation, and prevention. The thank-you note was always a means to an end. The end hasn't changed.
Sources & further reading #
[Vulnerability reports are not special anymore](https://words.filippo.io/vuln-reports/)— words.filippo.io -
[CRITICAL: Vulnerable HTTP Report | The Shadowserver Foundation](https://www.shadowserver.org/what-we-do/network-reporting/vulnerable-http-report/)— shadowserver.org -
[Vulnerability reports are not special anymore – Kamal Reader](https://rss.boorghani.com/vulnerability-reports-are-not-special-anymore)— rss.boorghani.com -
Vulnerability — Latest News, Reports & Analysis | The Hacker News— thehackernews.com -
[Vulnerability Reports: Don't Skip Them - Jera](https://jerait.co.uk/articles/vulnerability-reports-dont-skip-them/)— jerait.co.uk
[Ji-ho Choi](https://www.devclubhouse.com/u/jiho_choi)· Security & Cloud Editor
Ji-ho covers the increasingly tangled overlap between cloud architecture and security, drawing on a background as a penetration tester to keep his reporting grounded in real-world attack paths. He never lets a vendor claim go unquestioned and insists that every buzzword come with a proof of concept.
Discussion 0 #
No comments yet
Be the first to weigh in.