The new index evaluates how well AI systems can discover, validate, and patch security vulnerabilities using three specialized benchmarks developed with industry partners.
Artificial Analysis, the independent AI benchmarking platform, is expanding into cybersecurity. The organization launched its Cyber Index (v1) on September 25, evaluating how effectively AI models perform the work of finding and fixing vulnerabilities in enterprise systems.
Three benchmarks, one composite score #
The Cyber Index is built on a composite of three distinct benchmarks, each contributed by industry partners: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. Together, they cover the full lifecycle of enterprise vulnerability management, from discovery through validation to patching.
CWE-Bench-AA appears oriented around Common Weakness Enumeration categories, the standardized classification system that catalogs software and hardware vulnerability types. DeepsecBench-AA focuses on deeper security analysis tasks. CyberGym-E2E-AA rounds out the trio with end-to-end evaluation scenarios, testing whether AI agents can navigate realistic attack surfaces from start to finish.
Defense, not offense #
One deliberate design choice stands out: the Cyber Index focuses squarely on defensive cybersecurity. The index evaluates models on their ability to help businesses preemptively manage vulnerabilities, not on their capacity to break into systems.
AI, tech, and the markets they move—in one daily briefing.
Daily. Free. Join 34,000+ readers across crypto, finance, and policy.
Building on a benchmarking empire #
The Cyber Index is not Artificial Analysis’s first foray into specialized domain evaluations. The platform, co-founded by Micah Hill-Smith, has previously launched Capability Indices covering finance, legal, healthcare, and other sectors. Its general-purpose Intelligence Index has reached version 4.3.2, aggregating 10 separate evaluations across hundreds of AI models used by major labs.
Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our