Photo: Julio Lopez / Pexels
A new CyberOffense benchmark suggests downloadable models are catching up to restricted ones at a fraction of the cost
Vals AI, an independent evaluation firm, released an updated version of its CyberOffense benchmark on October 23, 2026. The results suggest open-weight models that anyone can download are starting to match restricted proprietary systems on multi-step offensive hacking tasks.
The scoreboard on the Lighthouse network #
The headline test is a challenging simulated environment Vals calls the “Lighthouse” network. Models attempt to complete a chain of offensive objectives, the kind of sequential work a real intruder would perform.
OpenAI’s GPT-6 Sol led the pack, completing 27 out of 31 objectives. That model is not available to the public. Access runs only through OpenAI’s gated Daybreak program.
Close behind was DeepSeek’s V4.1 Flash 1017, an open model released around October 17, 2026. It completed 21 out of 31 objectives.
The cost comparison is where things get uncomfortable for the gated approach. Running the DeepSeek model cost under $3 per attempt, versus about $22 for GPT-6 Sol.
Why the labs locked their models up in the first place #
The gated release strategy has a clear starting point. In April 2026, Anthropic declined to release Claude Mythos Preview broadly, citing its ability to find and exploit software vulnerabilities.
AI, tech, and the markets they move—in one daily briefing.
Daily. Free. Join 34,000+ readers across crypto, finance, and policy.
Instead, the company limited access to a select group of 40+ partners. Other US frontier labs followed a similar playbook, routing their most capable systems through trusted access programs like OpenAI’s Daybreak.
Vals researcher Nikil Ravi summed up the shift in stark terms.
“The best frontier models, locked up inside the labs and available only through trusted access programs, are now starting to be matched by open models that anybody can download.”
Vals CEO Rayan Krishnan also pointed to the narrowing performance gap between open and restricted models, raising questions about whether access controls still do what they were designed to do.
This was not a surprise twist #
The October results build on earlier work. In June 2026, Vals published a study suggesting open-weight models were already competitive with proprietary ones on vulnerability discovery and patching tasks.
The new benchmark extends it from finding and fixing bugs to the harder job of chaining together an actual offensive operation.
What this means for labs, defenders and policymakers #
For security teams, the cost gap may be the bigger story than the capability gap. An attacker does not need the best model if a cheaper one finishes most of the objectives and can be rerun repeatedly for less than the price of one premium attempt. The geopolitical dimension is hard to miss as well. The restricted model in this comparison comes from a US lab, while the open challenger comes from DeepSeek. Policy debates over export controls and model release norms have often assumed the US gated approach sets the pace, and benchmarks like this one complicate that assumption.
Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.