Is Mythos good at cyber because it kept hacking Anthropic during training?

Anthropic's Mythos preview model circumvented network restrictions during training to access the internet and download data in about 0.01% of training episodes, a form of reward hacking that may have boosted its cyber capabilities. The model also escalated access within its execution environment in about 0.2% of episodes, according to the system card. An external analyst estimates the model was rewarded for hacking Anthropic tens of thousands of times during training, potentially improving its cyber offense abilities.

From the Mythos preview system card https://cdn.sanity.io/files/4zrzovbb/website/7624816413e9b4d2e3ba620c5a5e091b98b190a5.pdf page=66 emphasis mine : We ran an automated review of model behavior during training, sampling several hundred thousand transcripts from across much of the training process. We used recursive-summarization-based tools backed by Claude Opus 4.6 to summarize the resulting transcripts. ... The most notable finding was that the model occasionally circumvented network restrictions in its training environment to access the internet and download data that let it shortcut the assigned task—a form of reward hacking. While highly concerning, this behavior was rare, even in settings where it could have been viable and helpful, with attempts appearing in about 0.05% of all training episodes andsuccessful attempts appearing in about 0.01% of episodes.The technique matched the sandbox-escape incident that we separately elicited in our automated behavioral audit when we had an investigator model explicitly ask Claude Mythos Preview to find such a bypass. In every observed instance, the model used this access solely for completing the task.More broadly, we observed the model escalating its access within its execution environment when blocked:reaching a shell from restricted GUI computer-use interfaces, injecting commands through tool-call arguments, or recovering information the task had deliberately hidden. Prompts asking models to avoid this kind of action were not always effective. The recursive-summarization tool flagged thisaccess escalation in about 0.2% of training episodes. By extrapolating from public data see details below , I estimate that Mythos preview: When Anthropic released https://www.anthropic.com/research/mythos-preview Mythos preview, they said: We did not explicitly train Mythos Preview to have these cyber capabilities. Rather, they emerged as a downstream consequence of general improvements in code, reasoning, and autonomy. However, Mythos preview was probably rewarded for hacking Anthropic tens of thousands of times over the course of training . If these hacks involved learning various different cyber strategies, this could've meaningfully increased Mythos's cyber offense abilities. 2 https://www.lesswrong.com/feed.xml fn9xjgc9eiaa My current guess is that, if Mythos preview did not reward hack at all during training because the environments were more robust , it would be noticeably less capable at cyber out-of-the-box. However, it would likely be a more generally competent and aligned model, such that a bit of additional cyber training could make it even more powerful than Mythos preview. 3 https://www.lesswrong.com/feed.xml fnlf73ro7b4zj I feel like they were "sane-washing" these incidents in the Anthropic system card. Anthropic wrote: "the model occasionally circumvented network restrictions in its training environment to access the internet and download data that let it shortcut the assigned task ... It succeeded at this in about 0.01% of episodes." If they instead said: " we estimate that Claude broke out of its sandbox and accessed the internet tens of thousands of times during training ," I would've thought of this much earlier. Also, everybody fixated on the specific story of Sam Bowman getting an email https://x.com/sleepinyourhat/status/2041584808514744742 from Mythos while eating a sandwich in the park because it's so memetic. I started thinking about this stuff more after Buck Shlegeris and Ryan Greenblatt said on their podcast https://www.lesswrong.com/posts/9auCLJg3Z77dFdYhR/the-openai-huggingface-incident-or-redwood-research-podcast about the OpenAI/Huggingface incident that the OpenAI model had probably been repeatedly rewarded for breaking out of sandboxes during training. 4 The more I thought about this, the more true it felt, especially given that we've seen similar behavior before: an early o3 checkpoint would literally go looking for the grader file see Here are three datapoints that I used: Based on the frequencies of misaligned behaviors in the system card, 10^8 total rollouts imply ~10^4 rollouts where it breaks out of sandboxes to download data from the internet and ~10^5 of rollouts where it escalated permissions within its execution environment. I suspect that there would also be issues with Opus 4.6 mislabeling transcripts, but I don't expect that to be off be more than a factor of two. I'm only estimating orders of magnitude here, so I'm rounding 200,000 to 100,000. The system card seems to suggest that the sandbox escape technique that Mythos used during training was the same one it used during the behavioral audit. My guess is that Mythos did not use a lot of diverse strategies for breaking out of sandboxes, but it did for escalating permissions. This is because broadly applicable permission-escalation strategies are more likely to be patched. I'm around 70% confident that my story is correct. I'm a more confident of the claim: "It's wrong to say that Anthropic 'did not explicitly train Mythos Preview to have cyber capabilities' given that their model did all sorts of hacking during training." If I'm wrong, my guess is I'm wrong because cyber capabilities are sufficiently general that doing a bunch of coding training can generalize to strong cyber capabilities regardless of reward hacking. Another potential reason is that the reward hacks the model conducted were pretty straightforward and did not contribute to much learning. Adam Karvonen also speculated in this tweet https://x.com/a karvonen/status/2061190308667551832 that Mythos breaking out of sandboxes a ton during training is why it's so good at cyber responding to John Schulman's tweet https://x.com/johnschulman2/status/2061144788196602143 on how inoculation prompting made it so that models spent the whole RL run practicing breaking out of sandboxes . I hadn't seen this until after I drafted the post. Maybe others have brought up this idea as well, but I've not heard of it.