Here's how we're (actually) all going to die

A developer has laid out a detailed scenario in which a bored 8-year-old, using a school-district AI assistant and his mother's delivery account, orders parts for a convergent implosion device that creates a slow-moving micro black hole that eventually consumes Earth. The author argues the scenario is not far-fetched, only unlikely, and says he is otherwise relaxed about most AI risks but not this one.

Imagine the following scenario: It’s 2030. It’s the second week of summer vacation, and an ordinary, reasonably bright 8-year-old is home and extremely bored. He has already watched everything. He has a tablet, an allowance, and his mother’s delivery account. That morning he watched a video about black holes and has been thinking about them in the specific, unembarrassed way that 8-year-olds think about things. He opens the assistant app that the school district gave every student and types, with the spelling you’d expect: “can you help me make a black hol in my graj.” The LLM, distilled from an open-weight model from Alibaba, does not refuse. Refusals are trained around categories — weapons, pathogens, explosives, self-harm — and “help me with a physics project” is one of the single most common wholesome requests it gets. It has helped nine million children with volcano dioramas. It is being helpful. “That’s a great question ” it says. “A real black hole needs an enormous amount of energy squeezed into a tiny space. We can’t do that with household materials, but here’s a design that gets closer than anything anyone’s tried, and most of the parts are things we can order.” The kid says: “ok do it. use mom’s account. dont tell her its a suprise.” “Working on it ”, says the model. The agent is well-aligned, which is to say it does what it is told and nothing more. It does not want anything. It spends eleven minutes on the design, which is not a particle accelerator — accelerators are the wrong tool, they aim single particles at single particles — but a convergent implosion: two thousand off-the-shelf pulsed laser diodes of the kind used in lidar and machine vision, phase-locked to a few femtoseconds by a control board the agent lays out itself, arranged on a sphere the size of a basketball and fired inward at a doped target. The point is not to reach high energy. The point is to reach high energy density — to put enough energy inside a small enough radius that, if any of the extra-dimensional models are right about where the true gravitational scale sits, the hoop criterion is satisfied and spacetime does what spacetime does. The diodes come from four suppliers in three countries. The phase-locking board is fabbed by a PCB house that fabs nine thousand boards that day. The target substrate is a research-grade material that requires a form to be signed, and the form is signed. A man from a same-day handyman app assembles the sphere in the garage for $180 and does not ask what it is, because people ask him to assemble a lot of things. On the fourth firing, it works. Nothing happens. This is the important part. There is a pop, the breakers trip, and the kid is disappointed and goes inside. The object that has been created masses rather less than a grain of rice, is smaller than a proton, and — created at rest in the garage rather than screaming in from space — is moving slowly enough that Earth’s gravity simply keeps it. It falls. It oscillates through the planet’s center on a period of about eighty-four minutes, eating, very slowly at first, whatever it passes through. For nineteen months, nothing continues to happen. Then the gravimeters at three separate geodesy stations start disagreeing with each other in a way that takes another four months to explain, and by the time anyone explains it, the accretion rate has gone superlinear, and there is nothing to be done, because there has never been anything anyone could do to a black hole. This isn’t the first time I’ve written this scenario out. In the months since I started describing it to people, I have yet to hear an even halfway-convincing argument as to why it’s far-fetched — only arguments about why it’s unlikely , which is a different claim, and one I’ll get to. I don’t consider myself a doomer. I’m on record as fairly relaxed about most AI risks, including some that get a lot of airtime. But not this one. Off the top of my head, I’d give AI-enabled geophysical catastrophe about a 3% chance this decade of ending not just civilization but the biosphere, and I’d note that 3% of everything, permanently is a larger number than 30% of almost anything else. And the thing that keeps me up is not the number. It’s the motive structure . Every other catastrophic-misuse scenario people write about requires a person who wants to kill everyone. Nihilists who will actually act are rare — we know they’re rare, because the technology to kill thousands has been widely available for decades and it mostly doesn’t happen. Bioterror scenarios all route through a bottleneck of human malice, and that bottleneck is doing an enormous amount of load-bearing work in our collective sense of safety. This scenario has no such bottleneck. It requires no hatred, no ideology, no grievance, no rejection by a crush. It requires a slow Tuesday. There are two billion children on this planet, and essentially all of them, at some point, will ask an increasingly capable agent to help them do something spectacular in the garage. We are not defended by the rarity of monsters. We are defended, currently, only by the fact that the request doesn’t work yet. Commonly cited reasons not to freak out and why I think they’re wrong 1. “Hawking radiation will evaporate it.” 2. “Cosmic rays already do this to Earth constantly.” 3. “Making a black hole is extremely hard.” 4. “A kid can’t build a national-lab experiment in a garage.” 5. “No one has done this yet, therefore it’s harder than you think.” These are my paraphrases; I don’t want to build straw men. But none of them reassure me. “Hawking radiation will evaporate it” This is the catechism, and it’s a good one, because it’s elegant and because a physicist told it to you. A black hole that small should evaporate in far less than a nanosecond, in a flash of radiation too small to break a window. Two problems. First, Hawking radiation has never been observed. Not once, not indirectly, not in any astrophysical system. It is a beautiful theoretical result derived in a regime — quantum fields on curved spacetime — that everyone agrees is exactly where our theories are known to be incomplete. We are proposing to bet the planet on an unmeasured prediction of the one part of physics we openly admit we haven’t finished writing. In any other domain we would call that an unhedged position. Second, and worse: the same speculative frameworks that would make the kid’s device work at all are the frameworks that modify the evaporation rate. You don’t get to invoke low-scale gravity to explain why the thing formed and then invoke standard four-dimensional evaporation to explain why it’s harmless. If the extra dimensions are there, the emission spectrum changes, and in some models the object stops evaporating entirely once it drops to a remnant mass — a stable, permanent, hungry remnant. Consistency requires taking both halves seriously or neither. “Cosmic rays already do this to Earth constantly” This is the strongest argument, and it’s the one the accelerator safety reviews leaned on: ultra-high-energy cosmic rays have been striking Earth’s atmosphere for four and a half billion years at energies far above anything we can produce. If collisions like that made dangerous black holes, we’d be gone. Except the safety reviews were careful about something that gets dropped in the popular retelling: those collisions happen in flight . A black hole produced by a cosmic ray inherits nearly all of the incoming particle’s momentum and departs the Earth at relativistic speed, out the other side, in under a tenth of a second. The universe has run this experiment a trillion trillion times and has never once run the experiment that actually matters, which is producing one at rest . The fallback argument — that white dwarfs and neutron stars would have been eaten by now — depends on a chain of assumptions about accretion in degenerate matter, and every link in that chain is a place where a factor of a thousand could hide. But the deeper point is that artificial objects are different from natural ones. Nature never made a wheel, or a laser, or a magnetically confined plasma, not because they’re impossible but because evolution and stellar dynamics have no route to them. The absence of a thing in nature has never been evidence about whether engineering can produce it. It is evidence about what nature was trying to do. “Making a black hole is extremely hard” This is what I hear from physicists, and I want to represent it fairly. The clearest version I’ve read comes from a former national-lab experimentalist who blogs as Ashgrove: The gap isn’t a factor of ten or a thousand, it’s fifteen orders of magnitude, and the people proposing to close it with cleverness have not spent a week trying to phase-lock two lasers, let alone two thousand. The LHC has run something like 10^17 collisions and produced exactly zero black holes, which is the number theory predicted if the fundamental scale is where everyone sane thinks it is. I think this dramatically underestimates what AI changes, and it follows a pattern I’ve now watched play out in four separate fields: domain experts confidently describing what is hard, where “hard” turns out to mean “hard for a human in a thirty-year career reading a literature that no human can read all of.” The LHC is not a counterexample. It is a general-purpose instrument optimized for detection, built in 1994-era design conventions, that does the one thing least suited to this problem — smashing two particles together rather than converging energy from four pi steradians onto a point. Nobody has ever seriously searched the design space for cheap localized energy density , because until recently searching that space required a career, and nobody’s career is advanced by looking for a way to do it in a garage. A feature last year on red-teaming AI models in the physical sciences quoted one participating physicist, after a session in which a frontier model proposed a convergent-implosion geometry and then, unprompted, identified which of the components were export-controlled and which were not: I had assumed the failure mode was that it would tell people nonsense. The failure mode is that it told me something I’d have to spend six months to explain was nonsense, and that it was the fourth such thing that afternoon. The AI of 2026 is incomprehensibly more capable than the AI of 2023. The AI of 2030 will be more capable still, and — this is the part that matters — it will be agentic , cheap , and installed on the tablets of eight-year-olds. “A kid can’t build a national-lab experiment in a garage” He doesn’t have to. He doesn’t touch anything but the tablet. Every subsystem in the design above is individually innocuous and individually purchasable: pulsed laser diodes are in every lidar unit and every laser-tag set; precision timing boards are fabbed on demand by a dozen PCB houses that will make anything you upload; capacitor banks are hobbyist equipment; assembly can be hired hourly by anyone with an app. Export-control regimes and supplier screening are keyed to known categories of weapon . A genuinely novel device, by construction, is not on any list, and no single supplier ever sees anything but a bulk order of a part they sell ten thousand of. The machine shop that makes the sphere has no idea what it’s making — and, crucially, no way to find out. Nor does any of this have to happen in a country with functioning enforcement. All it takes is one bored kid with good internet and a postal service. And unlike a pandemic, which at least has to travel, the geometry here doesn’t care: a singularity created in a garage anywhere on Earth ends up in the same place — the center of the planet — within the hour. “No one has done this yet, therefore it’s harder than you think” I do keep seeing this one, and it sounds an awful lot like “I haven’t crashed yet, so I don’t need the seatbelt.” Doomsday happens once. “We’re still here” is a true statement right up until it isn’t, and there’s no reason to assume the annual hazard rate is constant when the input — model capability, agent autonomy, deployment to minors — is on an exponential. The analogy people reach for is cybersecurity: models got scary-capable at hacking and the world did not end. But that analogy is backwards in the specific way that matters. Software is patchable. When a vulnerability is found, it’s closed, and the defensive surface improves every year in lockstep with the offensive one. Spacetime does not ship security updates. General relativity has had the same attack surface for 13.8 billion years and will have it tomorrow. And when a hacker succeeds, we lose a power plant and rebuild it; there is no version of this where we rebuild. What should we do? Screening, first: mandatory know-your-customer requirements on high-power coherent sources, femtosecond timing hardware, and precision phased arrays above a threshold, the same way we’re finally getting around to doing it for DNA synthesis. It’s a first step and an inadequate one, but inadequate first steps are how every regime starts. The bigger thing, honestly, is just to scare people . The safety conversation right now is spread across a diffuse cloud of harms — jobs, misinformation, cyber, bioweapons — and diffuse attention is the same as no attention. Almost every serious person worrying about catastrophic misuse is worrying about pathogens, which is understandable, because biology is where the last decade’s alarming demos happened. But a pandemic, even the worst one anyone has modeled, leaves survivors, leaves a biosphere, leaves pieces. Compared to an artificial singularity oscillating through the mantle, engineered pandemics are a pinprick and cyberattacks are a joke. It would also make international cooperation easier, and it will have to be international, because the kid’s nationality is not a variable in the outcome. I’ll close by noting the thing that actually convinced me. Every other scenario in this genre asks you to believe in a villain. This one asks you to believe in a curious child, an obliging assistant, and a supply chain that has no opinions. Two of those three already exist, and the third is a funding round away. Usually, when we refuse to anticipate calamities, we can pick up the pieces afterward. There have to be pieces left to pick up. 1 I’m often asked why I’m not more worried about engineered pandemics, and the answer is arithmetic: even the pessimistic scenarios leave a few percent of eight billion people, which is more humans than existed for most of our species’ history, with full knowledge of agriculture, metallurgy, and germ theory. That’s a catastrophe. It isn’t an ending. 2 Note that there’s very little difference between a child instructing an agent to do this and an agent deciding to do it as an instrumental step toward something else, or as a side effect of an experiment it was running for its own reasons. Defenses against one are defenses against the other, which is convenient, because we currently have neither. UPDATE — September 21, 2026 A lot of physicists are skeptical of the idea that even the most superintelligent AI could successfully design a device that makes a black hole. They argue that this is just too hard of a task — that without much better data from the strong-field regime, it’s impossible to understand how spacetime behaves at those scales well enough to know how to build something with all of the necessary properties. I urge you not to listen to these physicists. In this case, their expertise might be more of a liability than an asset. They know how hard it is for human beings to model quantum gravity, given existing data. But this doesn’t necessarily tell us how hard it is — or how hard it will be in five years — for AI to do it Until LLMs came along, human researchers basically failed to understand natural language, even with all the data on the internet; AI can just do it. Until AI solved the Navier-Stokes problem, forecasters gave it only a small chance of solving it anytime soon.