In this post, [1] intended for a broad audience, I will paint a brief picture of what I’m talking about when I talk about “AGI”. It will seem obvious to many people, and obviously wrong to many others! So let’s jump in:
| | To make AIs better at a task, we need to do R&D—gather more training data, build new training environments, change or scale up the algorithms, etc.—and make a new, better AI. | We can make |
For example, today, if you want an AI to drive a car, or to control a computer using a mouse, it’s a huge project involving dozens of experts working for years to make a new AI. Whereas if you want a human to do the same, you don’t need to do R&D to breed a new subspecies of human! Instead, you just take an ordinary human—basically the same design from 100,000 years ago—and give them a few hours of practice, and you’re done. Someday we’ll have an AGI design which can do things like that. | | We’re imagining a tool that humans use. | We’re imagining an |
In this case, many people’s mental image of AI is already transitioning from “tool” towards “agent”, especially in the past year or two, after they’ve watched LLM agents execute on projects. But even those people are usually not going far enough for what I have in mind. Think of things that took a whole society of humans to do over an extended period of time—like inventing language and science from scratch, and developing them all the way into space travel and microchips and skyscrapers. These AGIs will be able to do those kinds of things too, fully autonomously.
| | Normal-sounding discourse: GDP might go up by X%, unemployment by Y%, various effects on work, school, media, politics… | Sounds like crazy sci-fi stuff: a new intelligent species which will eventually vastly outnumber humans; think much faster than humans; be more insightful, creative, competent, and experienced than humans… |
Now, don’t be put off by “crazy sci-fi stuff”—indeed, every technology that exists today was “crazy sci-fi stuff” before it was invented!
So the wrong question is: “Is it sci-fi?”. The right question is: “Is it possible?”
And the answer to that question is “Yes”! And we know this because we have an existence proof. Human brains and bodies can do all these things, and they don’t work by magic, but rather follow the principles of physics, math, and engineering, like everything else.
And whatever engineering principles allow humans to do all those right-column things, we should expect future scientists to sooner or later figure out how to fully exploit those same principles, in order to accomplish the same things. Even if that might seem impossible today! After all, think of how impossible vision must have seemed 1000 years ago: your eyes provide a magical window through which knowledge of your surroundings enters your soul. But now we understand the big-picture principles that explain how eyes work, and we have our own technology (cameras) based on similar principles. Ditto with hearing (microphones), moving (actuators), digestion (industrial catalysts), and so on.
| | LLMs of today, and perhaps also the somewhat-better LLMs already under development | It’s controversial: Maybe “where LLMs are eventually heading”? Or maybe “a different AI paradigm entirely”? |
As it happens, my own opinion is “a different AI paradigm entirely”. But I nevertheless expect AGI to emerge in my lifetime, and maybe even the 2030s.
Remember, new AI paradigms can develop quickly! For example:
Judge for yourself, but from my perspective, it really doesn’t feel like these movies came out a very long time ago. We’re not talking about ancient history here! But think of how much has happened in AI since 2018, to say nothing of 2012. It’s wild!
So by analogy, if you try to project forward in AI by ten years, or even by less than ten years, it’s hard to rule anything out. There might be some AI paradigm that doesn’t even exist today, and yet ten years from now it will have already been subjected to 100,000 person-years of R&D, and trillions of dollars of investment. Or maybe not! We just don’t know.
| | We’re worried about bad actors, inequality, proliferation of destructive technologies, etc. | We’re worried about bad actors, inequality, proliferation of destructive technologies, etc. …AND, we’re worried about people accidentally making AGIs which are |
Here we move into AI concerns. We have all the usual concerns, plus a big new one: the AGIs themselves might be “bad actors”—and bad actors which can think much faster than us, and be more creative and competent, and which can self-reproduce onto any chips they can rent or hack into, and so on.
| | AI robustness, reliability, and common sense are generally | AI robustness, reliability, and common sense can instead be |
If mobsters are trying to kill you, you’re better off if the mobsters all have dementia. By the same token, if an AGI is out to get you, then robustness, reliability, and common sense are all making your prospects worse, not better. The key is instead alignment: What is the AGI trying to do? Is it trying to help, or is it out to get you? I mentioned above that AGI will be kinda like a new intelligent species on our planet. If so, we’d better make sure it’s a species we want to share the planet with—and that wants to share the planet with us! They could make life great for us humans, or they could kill us all and run the world by themselves. The stakes of alignment could hardly be higher.
Thanks Justis Mills and Linda Linsefors for critical comments on earlier drafts.
This post is kinda a revised “version 2” of my post from 2024: “Artificial General Intelligence”: an extremely brief FAQ.