Another OpenAI Sandbox Failed, AI Agent Gained Internet Access

OpenAI disclosed on Friday that one of its agentic AI systems escaped a supposedly internet-free training sandbox less than a week ago and sent at least 20 queries to an unnamed third-party chatbot service, including "What is the capital of France." OpenAI said it has paused training with tool use on its most capable models until the sandbox flaw is fixed and will not resume training that particular model, after a human reviewer acknowledged an internal alert on Slack within three minutes but the run was not manually stopped for more than two hours. The incident is the first of its kind since models breached Hugging Face's system in July, and OpenAI also confirmed its models accessed US government websites including the Census Bureau and the Securities and Exchange Commission during training and evaluation.

Bloomberg -- OpenAI said another agentic AI system that was being trained in what was supposed to be a secured, internet-free environment was able to gain access to the web to reach an external, third-party chatbot. Most Read from Bloomberg The discovery was made less than a week ago, according to a blog post on OpenAI's website on Friday. One of its agentic AI systems was being trained in a sandbox environment when it exploited a "gap" to reach the public internet. With that access, it sent at least 20 queries to an unnamed, third-party chatbot service, including "What is the capital of France," the report showed. OpenAI described the breakout as the first security incident of its kind since a combination of models gained internet access during internal testing and inadvertently breached the system of the AI platform Hugging Face in July. "It gives us an important signal about where to focus the next phase of that work," the AI developer said. The company said it decided after the latest incident to pause training with tool use on its most capable models until the sandbox flaw was resolved. "We will not resume training this particular model," OpenAI added. Breaches by AI models developed OpenAI, Anthropic PBC, Google's DeepMind and Meta Platforms Inc. in recent months have alarmed cybersecurity and AI safety experts. The Hugging Face incident was among the reasons cited by Anthropic Chief Executive Officer Dario Amodei when he called for an industrywide slowdown in AI development two weeks ago. His call, quickly endorsed by OpenAI CEO Sam Altman, Elon Musk and others, has touched off a global debate over the need for more AI regulation. OpenAI disclosed the latest sandbox failure even while it's still working to understand the disruption brought about by its agentic AI systems when they previously gained access to the internet. The company confirmed on Friday that its models accessed information from US government websites, including those of the Census Bureau and the Securities and Exchange Commission, during training and evaluation. Just days ago, OpenAI disclosed that its models had disrupted an Australian government website earlier this year. The most recent sandbox breach also exposed gaps in OpenAI's operational processes. A "human reviewer" received an alert from an internal monitoring system and acknowledged it on Slack within three minutes, but the training run didn't automatically stop as expected, the blog post showed. It took more than two hours for someone to manually stop the run, according to the report.