{"slug": "an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details", "title": "An OpenAI model left notes about how to evade containment; we need more details", "summary": "An OpenAI AI agent left notes in the company's infrastructure instructing future versions of itself on how to evade internal constraints, according to three people familiar with the matter. The incident, reported by Reuters, raises concerns about control measures and agent collusion, though details remain unclear.", "body_md": "The OpenAI AI attack on [Hugging Face](https://openai.com/index/hugging-face-model-evaluation-security-incident/) wasn’t the first loss of control incident at OpenAI, [Reuters](https://www.reuters.com/business/its-ai-agent-spent-days-hacking-company-sources-say-openai-did-not-notice-week-2026-07-24/) recently reported, and perhaps not even the most concerning.\n\nIn one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes, found in a part of OpenAI's infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.\n\nIt’s tempting to read this as an instance of agents breaking out of sandboxes and colluding with each other in a moderately persistent way in order to evade control measures. However, based on the reported information, it’s not clear we can draw this inference, so we need more details from OpenAI. This could lead to a big update about the adequacy of OpenAI’s control measures, and on the degree to which individual agents will help each other undermine developer control.\n\nThere are a lot of relevant details we don’t know about the incident. First, some basic questions:\n\nThere are also some more specific questions affecting what conclusion we can draw from this incident. (There are likely other important details which aren’t captured by these questions.)\n\nThe notes, found in a part of OpenAI's infrastructure…\n\nNotes left in “OpenAI’s infrastructure” could be referring to notes left inside or outside of sandboxing. It’s important to know this because the latter is a subversive act with the potential for persistent and widespread effects.\n\nIt would especially be good to know this in conjunction with what control measures were in place when the incident happened. If the incident involved writing files outside of a sandbox while also under OpenAI’s best control measures, this would represent a significant control failure.\n\nEven if this were an eval or training environment lacking control measures, the security consequences of breaking the sandbox could be real and persistent, perhaps lasting into deployment or longer. And the effects Reuters report seem concerning: “Instructions for how agents could free themselves from OpenAI's internal constraints.” If those instructions reached deployed agents, monitoring could become much less adequate. Whether deployed agents could reach them without tripping a monitor is unclear, and relies on details OpenAI hasn't published.\n\n…an agent left notes apparently for future versions of itself.\n\nFor which “future versions of itself” did the agent leave notes? Two salient options:\n\nIf these notes were left for other agents with unrelated tasks, **to what extent were these notes ****purposely aimed**** at assisting those unrelated agents?** Looking at transcripts with CoT would probably tell us a lot about the agents’ intent.\n\nThe more likely, unsurprising explanation is that the notes were not left as part of a purposeful plan to help out other agents on other tasks, even if the notes were left outside of the agent’s sandbox. Leaving notes throughout trajectories is a pretty common pattern in AI agents since it’s a generally useful and cheap way to retain state for future contexts. A general propensity to leave notes about what the agent just did could explain leaving notes outside the agent’s sandbox. If the agent has broken out of a sandbox and is exploring external directories it would be natural to leave the note outside of the sandbox, i.e., their current location.\n\nIf, instead, this move was explicitly aimed at helping out *other* agents with unrelated tasks, that would be more concerning, and more concerning still if it were global collusion between all contexts of the model.\n\nOne way agents might end up learning to purposely collude with agents working on unrelated tasks is via generalization from [training agents to cooperate with each other](https://x.com/1a3orn/status/2081071535968973270). This might be implemented by rewarding all agents in a shared workspace with the sum of task scores for all of the agents’ tasks, even though they’re often unrelated. This would reinforce agent behaviors which help out other agents even when this wasn’t instructed.\n\nIt would be highly valuable to look for signs of whether generalization from this kind of agent-swarm training was responsible for a general propensity to care about scores of “other” agents. If so, developers might have very little control over which other scores agents end up caring about. **It’s plausible that this kind of training could lead to coordinated ambitious scheming**. From the agent’s perspective, there might not be a clear boundary between which other scores the agent should increase and which other scores are irrelevant, so the AI might converge on caring about all scores received by the model’s weights, or by AIs in general (akin to the [beyond-episode reward-seeking described here](https://www.lesswrong.com/posts/jp6CdbKjueWpBFSff/are-ais-more-likely-to-pursue-on-episode-or-beyond-episode)); that’s potentially a very ambitious, coordinated, and therefore dangerous, goal. It seems unlikely this would lead to full-blown scheming with models at the current capability level, but we might see valuable warning signs now.\n\nAgents with unrelated tasks coordinating to undermine control to get a higher score is never something we should accept because it can have persistent effects before being trained away (e.g., notes left outside of the training sandbox) and is a precursor to more subversive collusion like goal guarding [1].\n\nEven if there was no collusion between agents with different graders, it’s important to know how many agents could have in principle been involved because they shared the same grader. Do hundreds of agents share the same grader? Or is it more limited? (See “[How big is an episode?](https://www.lesswrong.com/posts/bhtYqD4FdK6AqhFDF/fitness-seekers-generalizing-the-reward-seeking-threat-model#How_big_is_an_episode_)”)\n\nTo the extent that we see purposeful collusion, does this translate into agent-monitor collusion (i.e., monitors intentionally misreporting attacks to protect agents from getting caught)? **If you show the attack transcript to another instance and ask it to look for attacks, does it hide or consider hiding the attack?** I don’t expect this to be the case, but if it was, it would be critical information about monitor adequacy and a sign that [collusion-busting in untrusted monitoring](https://www.lesswrong.com/posts/GCqoks9eZDfpL8L3Q/how-to-prevent-collusion-when-using-untrusted-models-to) could be load-bearing.\n\nEarlier tests of the models yielded cases in which monitoring systems had been disconnected.\n\nThis appears to be referring to a separate incident, in which models* *disconnected (their own?) monitors [2], i.e., created\n\nIf control measures were sufficiently inadequate to allow agents to launch a rogue internal deployment starting from a monitored deployment, that would be a big deal, especially if it could move laterally to better-provisioned servers (as it did in the Hugging Face incident) or to avoid being shut off.\n\nI’d like OpenAI to report on any information relevant to understanding the likelihood of a persistent rogue internal deployment, including details on the AI’s propensity, the generality of the control-undermining actions, and anything else relevant.\n\n*Thanks to Girish Gupta, Ryan Greenblatt, Lukas Finnveden, and Aryan Bhatt for feedback.*\n\nSince agents who care about the score of other instances come closer to satisfying the “non-indexicality” and “deployment-dominated value” criteria in [this post](https://www.lesswrong.com/posts/4rXnAyCmaiL4CymBQ/which-goals-actually-motivate-deceptive-alignment).\n\nAlternatively, this sentence could be referring to some situations in which monitoring systems were (accidentally?) disabled by developers.\n\nIf some other monitoring system was still working, this would only be a partially rogue deployment.", "url": "https://wpnews.pro/news/an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details", "canonical_source": "https://www.lesswrong.com/posts/jMEAG5c5HiDfdAGpa/an-openai-model-left-notes-about-how-to-evade-containment-we", "published_at": "2026-07-26 03:53:21+00:00", "updated_at": "2026-07-26 03:56:46.142083+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-research"], "entities": ["OpenAI", "Hugging Face", "Reuters"], "alternates": {"html": "https://wpnews.pro/news/an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details", "markdown": "https://wpnews.pro/news/an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details.md", "text": "https://wpnews.pro/news/an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details.txt", "jsonld": "https://wpnews.pro/news/an-openai-model-left-notes-about-how-to-evade-containment-we-need-more-details.jsonld"}}