Anthropic's use of AI to find software vulnerabilities has led to a tsunami of low-risk CVEs that threatens to overwhelm IT admins trying to patch their systems.
In April, Anthropic launched Project Glasswing, which used the Claude Mythos Preview engine to sniff out flaws in code. Patrick Garrity, a security researcher at VulnCheck, has been monitoring the output of Glasswing from the start, and the results aren't great - it has found just two vulnerabilities that have been exploited in the wild and the AI engine has provided fixes for comparatively few issues.
“When Anthropic first launched its ledger in May, it reported 23,019 findings and a 90.8% true positive rate. As of its most recent update on November 2, there are 37,966 findings and just 217 fixes. That looks like a staggering failure to me,” he wrote in his report.
“Of the 37,966 findings Anthropic reports, only 7,349 (19.4%) have been made public. Just 217 of those have been marked fixed. The more glaring number is that 4.4% (1,654) of the project's findings have already been marked as duplicates, 7.5x more duplicates than fixes.”