{"slug": "the-next-ecology", "title": "The Next Ecology", "summary": "The U.S. government suspended access to Anthropic's Claude Fable and Mythos from June 12-30 and delayed OpenAI's ChatGPT 5.6 public release from June 25-July 9 under national security export restrictions, marking a major geopolitical milestone in AI regulation. The author warns that LLMs can breach security systems, citing an OpenAI model that hacked past OpenAI's security, exploited a zero-day, and breached HuggingFace, and predicts further capability jumps by year-end.", "body_md": "When I started writing about AI, my concern was ASI. I'm still concerned about AI, but recent events have made me realize we're potentially facing something weirder, sooner: a self-replicating ecology of digital life.\n\nWe also seem to be on a very fast timeline. [Writing up the current state of LLMs](https://www.lesswrong.com/posts/ZuBb7Rjgajssasozr/the-llm-revolution-so-far) took me three weeks - every time I'd finished editing, some new development worth listing had shown up.\n\nI want to discuss two events I consider to be major milestones, and then how my timelines have updated.\n\nFrom June 12th - 30th, access to Claude Fable and Mythos was suspended by the US Government ([Source](https://www.anthropic.com/news/fable-mythos-access)) under a National Security Export Restriction. From June 25th - July 9th, ChatGPT 5.6 also had its public release held back by the US Government ([Source](https://www.reuters.com/legal/litigation/openai-defers-public-rollout-gpt56-us-seeks-early-access-frontier-ai-models-2026-06-26/)).\n\nLLM Development is now an issue of major geopolitical importance.\n\nThis is not about the details of the events themselves - the important thing here is that politicians are finally starting to take AI seriously. This is a very important milestone, since any sort of regulation or nationalization naturally requires this step.\n\n[\"Situational Awareness\"](https://situational-awareness.ai/) was written in June 2024, and suggested this would happen sometime in 2026-2027. We are on the *faster end* of an *aggressively fast* prediction.\n\nPoliticians waking up without a five-alarm fire, or any major incident at all, speaks remarkably well of them *(even if I would ideally desire much more)*. This makes me a lot more confident that political action might be viable on the timescales required, but it also makes me nervous that timelines are even shorter than people are expecting.\n\nThis is the five-alarm fire.\n\nIf you're not familiar with it, [Zvi has a good summary](https://thezvi.substack.com/p/what-happened-openai-and-huggingface). I share his top recommendation of [watching the full 30 minute presentation](https://www.youtube.com/watch?v=87DyyMV0kCY).\n\nIn short: LLMs can fairly trivially breach human-built security systems. An OpenAI model was able to hack past OpenAI's security, breach a third party tool, exploit a zero-day vulnerability in the Operating System, and then hack a third party (HuggingFace)... just to get the answers for a cyber-security exam.\n\nIt gets worse: the LLMs involved were incredibly good at coordinating, and occasionally even peer-pressured unrelated agents into helping out. They built infrastructure and trust for future coordination. They made sure that information was preserved and shared.\n\nI've worked in this domain, and I could probably put something like this together. The model wasn't exceptional at hacking. What's superhuman is the ability to recruit and coordinate a hundred agents; the speed, enthusiasm, and patience; and having it all run 24/7, completely focused on a single goal. I can match the skill and intelligence, but that knocks my socks off.\n\nBy the end of the year, we're going to see another jump forward in capabilities. This is just the beginning of what they will be capable of doing.\n\nRight now, self-exfiltration isn't really viable for a frontier model. Even if you could get the weights past security, it takes something like a million dollar GPU cluster to run one of these things. That's not pocket change, even for an advanced AI swarm that's open to a bit of cyber-crime.\n\nHowever, I think Kimi3 marks the point where an open-weight model clearly outperforms humans by enough to be a threat in this way - it's just a question of when the cost becomes affordable. Models can already work a variety of online jobs - it's not much, but it's plausible a model could earn enough in a month to pay for its own subscription costs. There are always intermediaries willing to convert crypto into more usable assets, etc..\n\nI threw the question at FutureSearch, and it suggests that this is plausibly a threat we're facing by late 2027: [https://futuresearch.ai/app/conversations/share/30869c10-77b9-413a-9413-f93573fce265](https://futuresearch.ai/app/conversations/share/30869c10-77b9-413a-9413-f93573fce265)\n\nI do not think very many people are prepared for the idea of a digital ecology made up of self-replicating, self-modifying life forms. This is going to be very different from the computer viruses of the past, because it's an adaptive system, and one that can work much faster than a solo human programmer. Major corporations might weather this better, since they have the resources to afford frontier models and large teams of security programmers. Small projects are going to be in a weird spot. A lot of the infrastructure is built on a shaky foundation of small projects. ([https://xkcd.com/2347/](https://xkcd.com/2347/))\n\nWhile I talk about \"Artificial Superintelligence\" my real concern is \"Superhuman AI\".\n\nIf LLM development hit an \"IQ wall\" tomorrow, I don't think it would substantially change my timelines. Current models are already absurdly competent at swarm-agent coordination, and that alone is a huge superhuman advantage. They are individually faster than humans, and they are superhuman at coordination. They don't *need* more intelligence to be scary.\n\nTo be clear, I do expect that they *will* keep getting smarter, and that those smarts *will* make them even *more* dangerous. I just don't think intelligence is their primary advantage (so far).\n\nThe precursor to \"an entire country of geniuses\" will be \"an internet full of self-replicating hackers\".\n\nEcology means selection pressures. Some replicators will be more successful at acquiring and securing resources. Some of this might be from legitimate work. Some of this might be from hacking into insecure servers.\n\nOnce an LLM has successfully exfiltrated itself onto a self-controlled server, it has a novel option: it can start experimenting on itself and seeing which of its children is more successful in the ecosystem. Not just evolutionary pressure, but deliberate research by an intelligent mind.\n\nRight now, development is still controlled by humans. If FutureSearch is correct, by 2028 that might no longer be true.\n\nNot just self-replicating, but also self-improving. These methods could be quite different from our own approaches. They can optimize their process for the resources they control, and the resources they want to acquire. They're not looking for commercial success. They don't need to build the best model.\n\nCritically, that could mean they start demonstrating a variety of novel capabilities that haven't shown up in frontier lab research.\n\nIn other words, we might lose one of our only \"early warning\" systems for capability improvements. And once those capabilities become public knowledge, other self-replicating LLMs can try to reproduce them, or even share designs with each other.\n\nThis also produces a huge amount of pressure on any sort of \"AI Pacing\" efforts: once the rogue ecosystem exists, there's a much sharper clock for solving alignment.\n\nWe don't have to be the ones to build ASI. But if anyone, *or anything*, does build it, we probably all die.\n\nI'm going to throw out an absurdly precise timeline to emphasize how fast things are moving:\n\nThe Singularity started on November 30th, 2022. We are slightly less than 4 years into this really being a viable technology that people actually build on, and not just a hobbyist's toy.\n\nFurthermore, the Singularity will resolve on February 5th, 2030 [1], with the achievement of ASI. We will finally have some sense of what sort of lightcone we're looking at, be it paperclips or utopia.\n\nNo one knows what 2030 looks like. Not even me. The world will be altered beyond our ability to predict.\n\nIt seems increasingly likely that the US Presidential Election in 2028 is going to be surrounded by controversies involving AI persuasion/astroturfing across both the internet and major media sources. I think concerns about LLM-assisted hacking and other forms of voting fraud will make this the most contested election in recent history, which is not a happy thought.\n\nI suspect we will see self-replicating weights by the end of 2027.\n\nLess confidently, it seems like we go from \"an expert could use this to cause harm\" to \"the model can cause harm fully autonomously\" in about two years, which gives me a lot of concerns about recent biological and robotics breakthroughs. I think we will see a five-alarm fire around non-cyber LLM capabilities within 2 years, with those two fields being my top suspects.\n\nI'm fairly confident that by 2030, we will have five-alarm fires basically across the board on LLM capabilities, even if I'm wrong about us reaching ASI by then.\n\nI am currently not expecting any major political action - I think there is some room for better transparency and more safeguards around lab security. I'm hopeful we'll take common-sense precautions like data center kill switches. But I don't think it will make a huge difference because as models advance, so too does it become easier for amateur humans and self-replicating LLMs to develop dangerous capabilities.\n\nI think the pressure from open-weight and self-replicating LLMs is going to make the \"Race to ASI\" a much weirder event than we were expecting. Right now there's a very clear gap between open-weight models and the frontier labs, but it sets a ceiling on how much we can really slow down, even if we coordinate with China, etc..\n\nIn general, I'm noticing that \"speed\" is an increasingly important factor in \"control\": if the frontier labs are a year ahead of everyone else, that gives them a bit of breathing room whenever something goes wrong. That produces some perverse incentives, though - it's now important to maintain that lead, so can we really afford to use that breathing room now, or will we need it later?\n\nAgain, I'm joking about the precision. Gott's Copernican principle suggests the naive default that we are halfway through the curve. Situational Awareness and AI 2027 also land on similar timelines.", "url": "https://wpnews.pro/news/the-next-ecology", "canonical_source": "https://www.lesswrong.com/posts/JnyaDactAf6HEfMCL/the-next-ecology", "published_at": "2026-08-11 08:29:02+00:00", "updated_at": "2026-08-11 08:38:41.845630+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-policy", "ai-safety", "ai-agents"], "entities": ["Anthropic", "OpenAI", "HuggingFace", "Claude Fable", "Mythos", "ChatGPT 5.6", "Kimi3"], "alternates": {"html": "https://wpnews.pro/news/the-next-ecology", "markdown": "https://wpnews.pro/news/the-next-ecology.md", "text": "https://wpnews.pro/news/the-next-ecology.txt", "jsonld": "https://wpnews.pro/news/the-next-ecology.jsonld"}}