{"slug": "ai-vulnerability-patching-has-a-grading-problem", "title": "AI vulnerability patching has a grading problem", "summary": "Researchers at Northwestern University tested AI coding agents on 112 real security bugs and found that patches which stop a crash often fail the maintainer-written tests for the fix, with Claude Opus 4.7 producing patches that eliminated the crash but still failed those maintainer tests in a third of cases. The finding indicates that security patching benchmarks may overstate agent success by grading on crash elimination rather than maintainer acceptance criteria.", "body_md": "Researchers at Northwestern University tested AI coding agents on 112 real security bugs. They found that a patch which stops a crash often fails the checks the project’s maintainers wrote for the fix. With Claude Opus 4.7, third of the patches that made the crash go away still failed those maintainer tests. If you’re thinking about letting an AI tool patch vulnerabilities for you, this is the number to look at. Many security patching benchmarks … [More](https://www.helpnetsecurity.com/2026/10/12/ai-vulnerability-patching-grading-problem/) \n\nThe post [AI vulnerability patching has a grading problem](https://www.helpnetsecurity.com/2026/10/12/ai-vulnerability-patching-grading-problem/) appeared first on [Help Net Security](https://www.helpnetsecurity.com).", "url": "https://wpnews.pro/news/ai-vulnerability-patching-has-a-grading-problem", "canonical_source": "https://www.helpnetsecurity.com/2026/10/12/ai-vulnerability-patching-grading-problem/", "published_at": "2026-10-12 04:45:43+00:00", "updated_at": "2026-10-12 05:09:26.761625+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-research", "artificial-intelligence"], "entities": ["Northwestern University", "Claude Opus 4.7", "Help Net Security"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ai-vulnerability-patching-has-a-grading-problem", "markdown": "https://wpnews.pro/news/ai-vulnerability-patching-has-a-grading-problem.md", "text": "https://wpnews.pro/news/ai-vulnerability-patching-has-a-grading-problem.txt", "jsonld": "https://wpnews.pro/news/ai-vulnerability-patching-has-a-grading-problem.jsonld"}}