HAProxy Lead Willy Tarreau on AI Bug Finders: Hundreds of Reports, One in Three Fixes Merged HAProxy lead developer Willy Tarreau reported in a November 19, 2026 essay, "On the Use of AI-Based Tools to Find Bugs in Open Source Software," that AI tools now surface real bugs in open source but that maintainers absorb the review cost: HAProxy received hundreds of AI-assisted security reports over several months, most without a reproducer or patch, with about 300 to 400 from Claude Opus and GLM still pending review and only two or three looking serious at first glance. About one-third of the fixes Claude Opus proposed were merged after rework, and Tarreau named Claude Opus 5.5 as his most useful model despite frequent safeguard triggers, with GLM 5.3 finding roughly half the candidates at lower cost plus one real bug Opus missed. Tarreau, who wrote that the views are his own, also criticized safeguards firing on code he supplied, tiered access, and the Anthropic OSS scanner CLA. Willy Tarreau, the lead developer of HAProxy and a member of the Linux kernel security team, says AI tools now find real bugs in open source, and that maintainers are paying for it in review time. In an essay published on November 19, 2026, On the Use of AI-Based Tools to Find Bugs in Open Source Software https://www.haproxy.com/blog/on-the-use-of-ai-based-tools-to-find-bugs-in-open-source-software?ref=explainx , he reports hundreds of AI-assisted security reports over several months, most without a reproducer or a patch. He writes that the views are his own, not those of his company or community. The essay matters for anyone who runs open-source code in production or points a model at someone else's repository. It gives a maintainer's numbers, a model-by-model verdict, and a complaint about how vendors gate access. It follows the same pattern we tracked in GitHub and Microsoft: 9,791 AI Findings Became 99 Verified OSS Reports https://explainx.ai/blog/github-microsoft-mdash-9791-ai-findings-99-reports-november-2026 and in Google's bug-bounty freeze over AI slop reports https://explainx.ai/blog/google-oss-vrp-bug-bounty-freeze-ai-slop-reports-october-2026 . TL;DR: what Tarreau reports | Question | Answer from the essay | |---|---| | How many AI reports arrived? | Hundreds over several months; about 300 to 400 from Opus and GLM still pending review | | How many looked serious? | Two or three at first glance | | How many fixes were merged? | About one-third of Opus-proposed fixes, after rework | | Best model for him? | Claude Opus 5.5, with safeguards still triggering often | | Cheaper options? | GLM 5.3 about half of Opus 5.5's candidates, plus one real bug Opus missed ; local Qwen3.8-Flash-Next; Mistral Large 4 first test on par with GLM | | Main complaint? | Safeguards firing on code he supplied, tiered access, and the Anthropic OSS scanner CLA | What kind of reports is HAProxy receiving? Tarreau describes a flood, not a trickle. Over several months HAProxy received hundreds of AI-assisted "security reports." Most arrived without reproducers or patches, which makes each one expensive to judge. A maintainer has to decide whether the claimed bug exists before deciding whether it matters. Two batches came from Anthropic's Mythos model, each with 30 to 50 entries. He did not run Mythos himself. A colleague with access analysed HAProxy and wrote validated patches, and Tarreau says he was "genuinely impressed by the depth and complexity" of some of those bugs. The batches themselves were another matter. He says they lacked code excerpts, clear explanations, reproducers, fixes and commit IDs, which made them hard to use. The pile that remains is large. He counts about 300 to 400 AI-generated reports from Opus and GLM waiting for review, and says only two or three looked serious at first glance. For fixes, the picture is friendlier: about one-third of the fixes Opus proposed were merged after rework. That ratio is a useful benchmark. It means a model-written patch is a draft, not a finished change, but a draft that saves real time. For background on Mythos itself, see our Claude Fable 5.2 and Mythos 5.2 launch coverage https://explainx.ai/blog/claude-fable-5-2-mythos-5-2-launch-pricing-api-2026 . Which models did he use, and how did they compare? Tarreau names the tools he tried. The list is a snapshot of what a working maintainer reaches for in November 2026. Open source maintainer guiding a stream of AI bug reports in an AI bug finding workflow | Model or tool | His verdict | |---|---| | Claude Opus 4.8, Opus 5, Opus 5.5 | Used to decode Mythos reports, find bugs and write regression tests; Opus 5.5 was the most useful, though safeguards triggered often | | Claude Max 20x | Supplied through a six-month Linux Foundation subscription, used for agentic code review | | Fable 5 | Refused security-related work almost immediately; unsuitable for low-level development in his view | | GLM 5.3 | Cheaper and faster; found roughly half the candidates Opus 5.5 found on a first pass, but caught at least one real bug Opus missed | | Qwen3.8-Flash-Next with Strata | Run locally on a work GPU; "very satisfying results" | | Mistral Large 4 | First test at least on par with GLM | | OpenAI models and the Daybreak program | Access refused twice without explanation | | Anthropic OSS scanner | Noted as a new project, but criticised | Two lessons stand out. First, no single model covers everything: GLM found fewer candidates but one that Opus did not. A maintainer with budget for two models gets more coverage than one with budget for the best model twice. Second, a local model is viable for this job. If you maintain code you cannot send to a hosted API, the Qwen result is the part to read twice. We covered the same model family in NVIDIA's Qwen3.8-Flash-Next NVFP4 release https://explainx.ai/blog/nvidia-qwen3-8-flash-next-nvfp4-qad-quantized-multimodal-november-2026 . For how the frontier models compare on cost and routing, see Claude Fable 5.2 vs Opus 5.5 https://explainx.ai/blog/claude-fable-5-2-vs-opus-5-5-benchmarks-cost-routing-which-model-2026 . Why do safeguards get in the way? The sharpest part of the essay is about refusals. Tarreau says models sometimes refused to study root causes even when a working reproducer already existed. He asks: "What's the point of writing a candidate bug fix, compiling to test it, and suddenly claiming 'I'm not allowed to attack this code'?" His request is narrow. Safeguards should not trigger when a model works on source code the user provides. He lists what developers need: scanning code for bug candidates, writing reusable reproducers, writing several fix candidates in one session, eliminating false positives and writing regression tests. He lists what they do not need: weaponized exploits, system penetration, privilege escalation, data exfiltration or integrity attacks. This is a real design tension. A model that can write a reproducer for a memory bug is, by construction, close to a model that can write an exploit. Labs tune refusals to the risk of the second, and maintainers feel the cost of the first. We saw the lab side of this trade-off in Anthropic's coordinated vulnerability disclosure timelines https://explainx.ai/blog/anthropic-coordinated-vulnerability-disclosure-30-day-timelines-2026 and in OpenAI's Daybreak Red launch https://explainx.ai/blog/openai-daybreak-red-team-defense-factory-launch-november-2026 , which gates access by customer type. Tarreau criticises exactly that gating. He argues tiered access favours corporate and professional security customers over individual maintainers, who do much of the patching. His recommendation for vendors is developer-focused access for public projects, perhaps through a small subscription, with safeguards tuned to allow work on supplied source. He suggests GitHub or Microsoft could offer such a service. What does he say about the Anthropic OSS scanner? Anthropic's opt-in vulnerability-finding service for open source https://www.anthropic.com/research/launching-opt-in-vuln-finding-service-for-open-source?ref=explainx is the project Tarreau singles out. The scanner's repository https://github.com/anthropics/oss-scanner?ref=explainx is public, and our own post Anthropic OSS Scanner: Free AI Vulnerability Scans for Open Source Maintainers https://explainx.ai/blog/anthropic-oss-scanner-free-ai-vulnerability-scans-open-source-maintainers-2026 explains how it works. His objection is the contributor license agreement. He raises three points: - Section 2, copyright. A broad, perpetual license lets the company do nearly anything with submitted code. He suspects model training is a motive. That is his suspicion, not a stated Anthropic purpose. - Section 3, patents. He expects it to deter enterprises, which may have to give up patents or pay for a commercial model. - Section 4, legal entitlement. Signers must represent that they can grant the license, which he questions for contributions written by other people. He asks whether Anthropic got legal advice before doing this, and notes that several hundred developers have signed. Anthropic's position is not in the essay, and we have not seen a response. Until one appears, read this as a maintainer's legal worry, not a finding of fault. If you maintain a project, have whoever signs for it read the agreement first. Security patch on a cracked gear representing AI bug finding fixes in open source Threat models, and the numbers behind the flood Tarreau says HAProxy and the Linux kernel published threat models so that LLMs can calibrate severity. He reports that this reduced low-value reports, though the kernel still receives reports against outdated versions. A threat model states what the project considers a vulnerability, so a tool can discard findings that fall outside it. It costs a maintainer an afternoon and saves months of triage. He also says Anthropic's data seems to show the same pattern: only 0.5% of 7,300 vulnerabilities have been prioritized and fixed by open-source maintainers. We have not re-derived that number, and he quotes it as context rather than a finding of his own. Our VulnCheck ledger post on curl and the OSS scanner https://explainx.ai/blog/vulncheck-glasswing-november-ledger-curl-oss-scanner-2026 tracks related figures. The point is the gap: finding bugs is getting cheap, fixing them is not. What should different readers do? If you maintain a project. Tarreau's advice is to prepare for a flood of reports, good and bad, and budget time to review them. Publish a threat model. Say which versions you support. Ask reporters for a reproducer and a candidate fix. If you use AI to find bugs in other people's code. Do not send raw model output. Send a reproducer, a fix, a regression test and the exact commit you tested. Check the report against the project's threat model first. A third of Opus fixes merged only after rework, so expect to rework yours. If you pick a vendor. Test the refusal behaviour on your own code before you commit. Tarreau's experience is that two models with similar benchmark scores behaved very differently on security work. Try a second vendor and a local model. His concrete suggestion is to explore alternatives, including local ones. If you run open-source dependencies in production. Expect more advisories and faster patch cycles. Our post on the patch rush and outages Congress is asking about https://explainx.ai/blog/ai-vulnerability-patch-rush-outages-warner-congress-november-2026 covers the operational side. What is claimed and what is verified? | Claim | Source | Status | |---|---|---| | 300 to 400 AI reports pending at HAProxy | Tarreau's essay | His own count; not independently audited | | One-third of Opus fixes merged after rework | Tarreau's essay | His own project; one maintainer, one codebase | | GLM found about half of Opus 5.5's candidates | Tarreau's essay | A first-pass comparison on his code, not a benchmark | | 0.5% of 7,300 vulnerabilities fixed by maintainers | Anthropic data, as quoted by Tarreau | Quoted without a link; not checked here | | The OSS scanner CLA has the problems he lists | Tarreau's reading | Legal opinion; no Anthropic response seen | | Fable 5 refuses almost all security work | Tarreau's test | His experience on low-level code | Bottom line The essay is valuable because it comes from someone who reads every report. AI bug finding works well enough that a security maintainer keeps paying for it. It also works unevenly enough that the cost lands on the reader of the report. The practical answer is not to stop. It is to send fewer, better reports, publish a threat model, and demand that vendors treat a maintainer working on supplied source code differently from an attacker. For the wider picture of AI and software security, read Human slop: AI exposing decades-old vulnerabilities https://explainx.ai/blog/human-slop-legacy-code-ai-exposing-decades-old-vulnerabilities-2026 . Related reading Details reflect Tarreau's essay of November 19, 2026 and may change as vendors respond.