CVE-Bench

The Center for AI Standards and Innovation (CAISI) released a repository packaging 2 agents and 3 benchmarks for agentic cyber capability evaluations of AI systems, built on the Inspect evaluation fra…

A new open-source benchmark called CVE-Bench evaluates large language model agents on fixing real-world security vulnerabilities by running them inside sandboxed Docker containers and scoring them aga…

Researchers evaluated five frontier AI models (three from OpenAI, two from Poolside) on fixing 20 real-world Common Vulnerabilities and Exposures (CVEs) across three prompt conditions, finding that no…