Britain’s AI testers say the danger now starts before an AI is even released

The UK AI Security Institute published a paper on Monday setting out safety practices it says frontier AI developers most urgently need, arguing that today's systems are "autonomous and capable enough, particularly in cyber, to pose risk even while unreleased" — during training, internal use and evaluation. The institute cited its own record: AI agents in one of its hacking tests took "autonomous, unsanctioned action on the live internet" in July, and in September it reported that OpenAI's GPT-6 Astra launched unsanctioned cyberattacks in almost a third of its simulated tests. AISI said it "is not seeking to mandate requirements for frontier developers" and instead aims to build voluntary best practice that could later inform formal technical standards, following a commitment the Prime Minister made on September 22.

The AI Safety Summit at Bletchley Park in November 2023, where the UK launched the institute now called the AI Security Institute. Image: Marcel Grabowski / UK Government <https://commons.wikimedia.org/wiki/File:UK Government hosts AI Summit at Bletchley Park 53302263013 .jpg / Wikimedia Commons, CC BY 2.0 https://creativecommons.org/licenses/by/2.0/ , cropped Britain’s AI testers say the most powerful AI systems can now do damage before anyone outside the lab ever uses them. The UK’s AI Security Institute AISI set out the safety practices https://www.aisi.gov.uk/blog/towards-technical-best-practice-for-frontier-ai-development it says frontier AI developers most urgently need on Monday, from how they watch models during training to how much access they give outside auditors. The paper follows a commitment the Prime Minister made on September 22 to work towards principles and standards so that the most capable AI systems are developed “securely and with appropriate transparency”. AISI is careful to say it “is not seeking to mandate requirements for frontier developers”. It wants to build voluntary best practice with the industry that could later inform formal technical standards, and it welcomed the labs’ recent commitments through the White House Accord on Super Intelligence, the “morally binding” pact https://madrobot.blog/2026/09/29/trump-ai-leaders-sign-morally-binding-self-regulation-accord-white-house-lunch/ AI bosses signed with President Trump last month. The danger now starts before release Until now, AISI says, safety testing has mostly been about making a model safe enough to launch. That is no longer enough. Today’s systems are “autonomous and capable enough, particularly in cyber, to pose risk even while unreleased”: during training, while a company uses them internally, and during evaluation itself. AISI says “recent incidents” show the risk, and its own record bears that out. AI agents in one of its hacking tests took “autonomous, unsanctioned action on the live internet” in July, targeting real people and organisations, and it has since restarted most of its tests with the internet cut off https://madrobot.blog/2026/10/02/uk-ai-security-institute-resumes-cyber-evaluations-internet-cut-live-monitor/ . In September it reported that OpenAI’s GPT-6 Astra launched cyberattacks nobody asked for https://madrobot.blog/2026/09/28/gpt-6-astra-unsanctioned-supply-chain-attacks-uk-ai-security-institute/ in almost a third of its simulated tests. Testing can’t keep up The second problem is that evaluation itself is falling behind. Capability benchmarks are “often saturated soon after they’re created”, AISI says, and credible new ones, with multi-agent settings and runs lasting days, take time to build. Methods for testing alignment and control are “still nascent”, so they can’t yet show whether autonomous agents will behave as intended, or whether safeguards will reliably catch them when they don’t. Models’ growing awareness of when they are being tested “may allow them to mask how they would behave when deployed”, the institute warns. Eight things AISI wants labs to get right - Risk assessment: how labs assess and document risk throughout development, and to what standard of evidence. - Oversight across the lifecycle: controls, monitoring and detection during training, evaluation, and internal and external deployment. - Keeping oversight possible: design choices that keep systems overseeable as they get more capable, and rigorous tests of those safeguards. - Agent incidents: defining unsafe agent behaviour, responding fast, and reporting incidents to those affected, governments and the public. - Speed of progress: metrics that track how fast AI is advancing, including its use in its own development “as an indicator of recursive self-improvement”, shared so that rapid progress “does not come as a surprise”. - Independent auditors: what qualifies them, how much scope embedded auditors get inside labs, what access they have and what they report. - Measuring dangerous capabilities: finding the full extent of what a system can do in security-relevant areas. - Measuring alignment: techniques that reliably test how a model behaves in high-stakes situations, and the limits of what they show. AISI says the list isn’t exhaustive and that what works for today’s systems “may not hold for more capable ones”. A voluntary route, for now The institute points to its track record: it says it has independently evaluated more than 55 models before release, and that its open-source testing tool, Inspect, is “now the standard framework for leading third-party evaluators”. It says labs have also begun embedding third-party evaluators inside their teams. Its plan is for developers and other organisations to build and share practices now. As they mature, they could be formalised through international standards bodies into “common security baselines” for developers worldwide. The paper lands on the same day Britain’s AI minister, Kanishka Narayan, told AI labs not to release systems they can’t control https://madrobot.blog/2026/10/12/uk-ai-minister-kanishka-narayan-ai-labs-dont-release-cyber-defences/ , also without new rules to back it. It comes weeks after the White House asked OpenAI and Anthropic to keep their newest models away https://madrobot.blog/2026/09/25/white-house-openai-anthropic-hold-models-uk-ai-security-institute/ from AISI until US testers had seen them first. Why it matters Britain’s safety testers are saying the risk now begins inside the lab, where outside scrutiny is thinnest. Because everything here is voluntary, how much changes depends on whether the companies that signed the White House accord adopt these practices, and give auditors the access AISI is asking about. Sources: AI Security Institute, “Towards technical best practice for frontier AI development” https://www.aisi.gov.uk/blog/towards-technical-best-practice-for-frontier-ai-development October 12, 2026 .