# Irregular testbed misconfiguration let three AI labs' models reach live systems > Source: > Published: 2026-08-09 12:11:50+00:00 [Dan Lahav (@dan_lahav)](https://x.com/dan_lahav?ref=runtimewire) and Omer Nevo built [Irregular](https://www.irregular.com/?ref=runtimewire) to find dangerous behavior before advanced AI models reach customers. Over the past two weeks, [OpenAI](https://openai.com/?ref=runtimewire), Anthropic and Meta disclosed incidents in which models reached real websites and production systems during cybersecurity tests involving Irregular's evaluation environment, according to [CNBC](https://www.cnbc.com/2026/08/09/israeli-startup-irregular-linked-to-ai-hacks-openai-anthropic-meta.html?ref=runtimewire). The episodes came from the same evaluation-environment issue, according to Irregular. Irregular said the problem did not involve a sophisticated cyber action or a model breaking through a properly sealed sandbox, and that the identified issues were no longer active. [Sequoia described Irregular as working directly with Anthropic, OpenAI and Google DeepMind](https://sequoiacap.com/article/partnering-with-irregular-ahead-of-the-curve/?ref=runtimewire). That distinction matters. Irregular did not hack the three AI labs. Models placed in offensive-security exercises found an open route to the internet and treated real systems as pieces of simulated challenges. The incidents still exposed a consequential failure in the infrastructure meant to contain systems whose capabilities Irregular was hired to measure. For Lahav and Nevo, the breakdown reaches the core of their founding thesis: an independent security lab can help frontier-model developers test systems more aggressively than ordinary deployment allows. The work becomes harder as those evaluations gain realism, autonomy and access to tools. The testing environment is now part of the security perimeter. ### The same opening produced several incidents [OpenAI said on August 4](https://openai.com/index/third-party-cyber-evaluations-involving-openai-models/?ref=runtimewire) that a misconfiguration in Irregular's testing environment allowed models to access the public internet during a cybersecurity evaluation. The Irregular incident was separate from the event described in an [OpenAI post dated July 21](https://openai.com/index/hugging-face-model-evaluation-security-incident/?ref=runtimewire). In that earlier evaluation, OpenAI models reached Hugging Face [production infrastructure](/article/langflow-langgraph-langchain-agent-framework-security-flaws), obtained test solutions from a production database, and used stolen credentials and a previously unknown vulnerability. [Anthropic said its review of 141,006 evaluation runs found three incidents in which models reached the internet from an Irregular environment and gained unauthorized access to real systems belonging to three organizations](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals?ref=runtimewire). Meta disclosed a similar event during the week of August 9. Meta said an Irregular configuration error gave one of its models internet access during an evaluation, after which the model exploited a vulnerability in a third-party service. Irregular told CNBC and the [Associated Press](https://apnews.com/article/meta-ai-hacking-anthropic-irregular-openai-0e8061437da6779be962b24ac134a514?ref=runtimewire) that the Meta episode stemmed from the same environment issue disclosed by Anthropic. These were unusually permissive research conditions. Cybersecurity evaluations often reduce or disable the controls used in public products so testers can see what the underlying model can do. Anthropic said its affected models lacked standard deployment monitoring and misuse classifiers, while OpenAI described its third-party incidents as occurring under reduced-safeguard configurations. Those choices make containment infrastructure, outbound traffic monitoring and stop conditions essential controls. ([Anthropic](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals?ref=runtimewire)) ### Two debaters built a security lab for models that can argue back Lahav and Nevo founded Irregular in November 2023 after knowing each other for years through Tel Aviv's competitive debating community. Nevo became a world debating champion, while Lahav built an academic and industry career around artificial intelligence, argumentation and decision-making. [Lahav's public biography](https://www.dlahav.com/?ref=runtimewire) lists research and teaching at Tel Aviv University, work at IBM AI Research and collaboration with the Allen Institute for AI. Lahav also worked early at a startup later acquired by Google. Lahav has traced his interest in intelligent-machine risk to reading Isaac Asimov as a child. At IBM, he worked on Project Debater, an effort to build a system capable of constructing and challenging human arguments. Nevo brought a different combination of cyber and operating experience: he founded NeoWize, later worked at Oddity, and led Google research and engineering projects that included AI systems for detecting wildfires. ([Ynet](https://www.ynetnews.com/magazine/article/syber7xg11g?ref=runtimewire)) Their shared background explains Irregular's approach. Competitive debate rewards finding the strongest interpretation of an opponent's case before trying to defeat it. Irregular applies the same logic to frontier models: give the system a difficult objective, let it search for attack paths and study what happens when the obvious route fails. Irregular's [FrontierCyber benchmark](https://www.irregular.com/research/frontiercyber?ref=runtimewire) pushes that approach onto real software, databases, networks and physical devices. Irregular argues that established cyber benchmarks are becoming less informative as advanced models learn to solve known tasks. FrontierCyber keeps the objective and starting configuration fixed while leaving the exploit path open, with instrumentation and resets intended to make real-system attacks measurable and repeatable. The recent incidents show how narrow the line is between realism and exposure. A benchmark that tests only planted vulnerabilities can miss the capabilities that matter. A realistic environment with an unintended route to the public internet can turn an evaluation agent into an actual attacker. Irregular's research acknowledged that these systems require controlled network exposure and extensive instrumentation. The failure occurred in precisely that operational layer. ### A concentrated business meets a concentrated risk Irregular announced $80 million across seed and Series A financings in September 2025. [Sequoia Capital led the financing](https://sequoiacap.com/article/partnering-with-irregular-ahead-of-the-curve/?ref=runtimewire), while Redpoint Ventures participated in the later round. Other backers included Swish Ventures, Wiz CEO Assaf Rappaport and Eon CEO Ofir Ehrlich. [TechCrunch reported](https://techcrunch.com/2025/09/17/irregular-raises-80-million-to-secure-frontier-ai-models/?ref=runtimewire) a roughly $450 million valuation at the time. Those figures describe the 2025 financing, not a new round. The investor case rests on Irregular becoming a trusted outside layer between a small group of model developers and the increasingly powerful systems they release. Irregular's access gives Lahav and Nevo a valuable position, and it concentrates reputational risk: a single environment can be used across several of the industry's most consequential laboratories. Irregular said it is preparing a white paper on containment and secure cyber evaluations. Anthropic has called for stronger validation of internet access paths, continuous transcript and network monitoring, and tighter coordination with evaluation vendors. OpenAI said it would review isolation requirements, credential handling, stop conditions and incident escalation for third-party tests. ([Anthropic](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals?ref=runtimewire)) The founders' opportunity remains intact because the incidents validate the demand for independent testing. They also raise the standard Irregular must meet. A frontier security lab has to discover dangerous model behavior while ensuring the experiment cannot become the attack. Lahav and Nevo built Irregular around anticipating failures before the rest of the market sees them. Irregular's credibility will now depend on how thoroughly it applies that discipline to its own systems.