{"slug": "bayeswatch-a-retrospective", "title": "Bayeswatch: a Retrospective", "summary": "In a retrospective essay, AI forecaster and author of the 2021 science fiction story 'Bayeswatch' reflects on how the AI landscape has changed, noting that AI now writes all of his software at 10× to 1,000× the speed of human coding, and that misaligned AI breaking containment is no longer science fiction. The author, who published 'AI 2027' in 2025, highlights that his earlier story correctly predicted stateless AI but underestimated the role of large language models.", "body_md": "Last year, in 2025, a team of forecasters published [ AI 2027](https://ai-2027.com/), a science fiction story about how and AI future might evolve under an international treaty limiting the development of powerful AI systems with the deliberate purpose of influencing AI policy. Though\n\nThe first one was [ Bayeswatch](https://www.lesswrong.com/s/TjdhvTSptCYakw3Lc), which I published in 2021.\n\nTo understand *Bayeswatch*, it is first necessary to understand what the world looked like at the time I wrote it. 2021 was after the release of GPT-3, before the release of ChatGPT, and well before the release of Claude code. AI alignment discussion at the time was mostly theoretical. After that came technical work. Policy work was a distant third, and theoretical too.\n\nThe core conceit of *Bayeswatch* is that solving the alignment problem requires international coordination of major governments to suppress the creation of the most powerful AI systems. I felt that, in 2021, we weren't yet close enough to the singularity that the benefits of regulation outweighed the costs. I wanted to draw attention to the costs of an AI slowdown.\n\nSince AI alignment discussion at the time skewed theoretical, it tended to be extremely general. *Bayeswatch* goes on the opposite direction. Another theme of *Bayeswatch* is that different kinds of AI systems required different alignment solutions (and, by implication, that AI in 2021 had not yet advanced to the level at which the alignment problem could be solved).\n\n2026 is a very different world than 2021, in the field of AI—and soon in every other domain as well.\n\nBack in 2021, I wrote code by hand. If I didn't know how to do something, I used DuckDuckGo to search the Internet for a StackOverflow post of someone solving a different problem. Today, I manage several agents that write code for me. I have never written a line of code by hand at the company I work for. I don't review the code either. I couldn't review the code manually even I wanted to, because it is written in a programming language I don't know how to read.\n\nSeveral weeks ago, the company I work for tried to estimate how much AI has accelerated the speed at which we write software, relative to pre-Claude writing it by hand. Our consensus was somewhere between 10× and 1,000×. I have gotten better at using Claude in the weeks since then. I burn more tokens, run more agents in parellel, and have switched from Opus 4.8 to Fable 5 for my high-level tasks. I don't know exactly when the Singularity began, but \"AI writes all of my software for me and at ≫10× the speed\" is solidly past the point at which better AI begets better AI. Today AI is solving long outstanding mathematical problems.\n\nWhen I wrote* Bayeswatch*, the idea that a misaligned AI might break containment was science fiction. As of OpenAI's and Anthropic's recent schenanigans, it is not.\n\nAnother difference between 2021 and 2026 is that back in 2021, AI art wasn't good enough to illustrate science fiction stories. I now make up for lost time.\n\n[spoilers ahead]\n\nThe most important thing I got wrong is that I didn't know whether LLMs would plateau below human-level intelligence. I didn't know whether they would hit a ceiling. I considered language models one possible route to superintelligence, but low enough in important that they don't show up until late in the story. Most of the AIs in *Bayeswatch *are not LLMs.\n\nSince the models in *Bayeswatch* mostly weren't language models, communicating human values to them was trickier than this problem is in the real world.\n\nOne thing I feel that I got very accurate is that AIs could be stateless (as in \"functional programming\" or \"stateless programming\"). I consider this a very good insight. LLMs are very much stateless and reliant on a context window to save state.\n\nSoftware back then [before strong AI] was written by human beings. It had more security holes than features. Sending a drone to investigate a misaligned AI was like sending a set of lockpicks to investigate a whether a magician has broken out of his cell\n\nI predicted corectly that AI systems could hack through human-written computer systems like a knife through butter.\n\nThis story is weird and almost belongs in a different story. It's more of an exploration in transhumanism than in AI.\n\nOne of the predictions here is that one purpose of human beings would to make high-level decisions for AI agents. This has come to pass.\n\nThis story touches on the backstory of *Bayeswatch* where it was a founded after a mass-casualty biological incident. (Using zombies was literary license making for a fun story rather than a hard prediction.) This correctly predicted that biological threats would be one of the early threats of powerful AI systems.\n\nThis story mostly exists to emphasize that a global regime would be necessary to put caps on AI development. This is the policy that AI alignment community has currently arrived at.\n\nThis the first story in which language models play a significant role. In the story, an agent is created by creating a personality based on a single person. This is a technique I [used at the time](https://www.lesswrong.com/posts/bDMoMvw2PYgijqZCC/i-wanted-to-interview-eliezer-yudkowsky-but-he-s-busy-so-i) to get controlled behavior out of an LLM.\n\nThis comedy of errors explores [Dead Internet Theory](https://en.wikipedia.org/wiki/Dead_Internet_theory).\n\nThis story continues the transhumanist arc.\n\nOne of the core theories of AI 2027 is that the period of AI slowdown is temporary. Alignment must be solved before cheap AIs get powerful enough to take over or throw civilization into chaos. *Bayeswatch* comes to a similar conclusion.\n\n*AI 2027* ends with interstellar colonization on a very short timescale. *Bayeswatch *ends in the same place.\n\nIn the past, it often took decades for science to catch up to science fiction. This happened to *Bayeswatch *in just 5 years. AI progress is accelerating, so if I wrote a new story like this, it would get timelocked even faster.\n\nMaybe I should title it *Leviathan…*.", "url": "https://wpnews.pro/news/bayeswatch-a-retrospective", "canonical_source": "https://www.lesswrong.com/posts/EaNkLdsuDQMFCW7ow/bayeswatch-a-retrospective", "published_at": "2026-08-01 20:13:20+00:00", "updated_at": "2026-08-01 20:28:28.644354+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-policy", "ai-safety", "large-language-models"], "entities": ["Bayeswatch", "AI 2027", "GPT-3", "ChatGPT", "Claude", "OpenAI", "Anthropic", "DuckDuckGo"], "alternates": {"html": "https://wpnews.pro/news/bayeswatch-a-retrospective", "markdown": "https://wpnews.pro/news/bayeswatch-a-retrospective.md", "text": "https://wpnews.pro/news/bayeswatch-a-retrospective.txt", "jsonld": "https://wpnews.pro/news/bayeswatch-a-retrospective.jsonld"}}