This is a timed post. Every 5 minutes while writing this post, I had to stop to do 13 push-ups, and when I could no longer complete my required number, I had to upload it. My friends thought this would be a fun challenge, but it means that it will likely be less polished than some of my other output.
Summary:
Many people moving into AI safety are told that to succeed, they should read a lot and “try to develop takes.” However, it’s important not to forget why knowledge is important: it helps you do better things in the real world.
*Epistemic status: Pretty confident in the main claim (you should focus on learning the most valuable things), but less confident in the specifics of what things are the most valuable. I apologize if my language is too strong at times. *
The world is filled with ambitious young people, some of whom care about AI risks. Throughout my summer, I’ve gotten to talk to many of them. They visit San Francisco and Berkeley to talk to all the cool AI safety workers, or they go on surprise trips to work on high-impact, time-limited opportunities. Throughout this process, many of them look for jobs, and if they can’t find them, they seek out advice on how to get good enough to land one.
A common piece of advice for how to get a job in AI safety is “read a lot and try to develop good takes.” This advice is featured very broadly, even for people who never intend to take a technical research role:
As this knowledge propagates, I worry that many people who hear this advice lose track of why it’s important. Many people sharing this advice don’t spend a lot of time explaining why it’s valuable for everyone in AI safety to have their own takes.
It’s obvious why you would want to know things if you’re trying to become a knowledge expert: knowledge experts are people who know things. However, if you’re trying to become a competent chief of staff, event manager, or field builder, or you want to take one of the many other neglected generalist or org scaling roles, it’s not as obvious why you should read everything. It’s good to have an explanation for why what you’re doing is impactful, but when I ask newcomers why they think it’s important to read, I often get answers like, “I dunno, it seems good to know things?”
I worry about this. When people don’t have a good picture of why they’re doing something, they might overestimate something’s impact and miss its end goal. If you want to do good things, this is important to avoid.
**Knowledge is valuable because it helps you make better decisions. **Orgs want people to be knowledgeable because people who know specific things about AI safety can do impactful things, not because storing more facts about AI is intrinsically valuable. You should focus on gaining the knowledge and tools that help you make better decisions.
To do this, I recommend using a process I call directed reading: identifying the most important questions that will help you make better decisions and doing everything you can to get answers to those. This doesn’t necessarily have to be reading: it can be listening to a podcast, talking to a friend, or chatting with an LLM. But the primary goal is to resolve your most important uncertainties.
Here are some examples of uncertainties you may have:
To answer many of your questions, you’ll probably need to find answers to smaller and more fundamental questions. On its surface, your process might seem to replicate the original “read a lot of things” advice. However, by focusing on what you’re actually hoping to achieve with your knowledge search, you can develop critical skills much faster than you would by passively consuming the next 80,000 Hours podcast episode.[2]
You can also double-check your learning processes. If the knowledge you’re getting doesn’t help you make better decisions, you should be willing to quickly pivot to trying other things. Learning things does not intrinsically save lives. Misalignment risk isn’t reduced by the number of books you’ve read from cover to cover. Don’t waste time memorizing social passwords and storing fake knowledge. [3] Do what will help you do good things in the real world.
While focused learning is important, I don’t think directed reading is the best way to solve all of your problems. First, while directed reading can be one of the best ways to resolve your largest uncertainties, you don’t need to spend all of your time skimming posts to find only the answers you need. Reading things you enjoy is pretty awesome, and you shouldn’t force yourself to spend all your time engaging with material that feels like homework. Find ways to integrate your study time into your routine, but don’t forget to do the things that make you happy.
Second, pure information consumption might not be the best way to gain skills and understanding. Writing can be a valuable tool to help you internalize and understand various ideas, as well as keep track of any uncertainties you might still have. [4] Additionally, if nobody’s written about the thing you want to say, you should consider polishing your notes and publishing them online. If it helps, you can assemble a group of friends and do a timed writing challenge to publish the scraps you have lying around.
Finally, you might be completely wrong about what you think you should be studying, or you might be missing some valuable new leads by continuously taking a focused approach to learning. To address this, it can be valuable to introduce some randomness into your media diet: find some time to read things (especially highly recommended things) that you don’t expect to be helpful. However, I suspect that most people aren’t spending enough time trying to learn things directly.
In the immortal words of the great Thomas Henry Huxley (aka “Darwin’s Bulldog”), “The great end of life is not knowledge, but action.”
I think I wasted about a week because I didn’t ask myself this question. These kinds of “minimal trust investigations” are valuable for becoming highly talented in a particular domain.
No hate to 80,000 intended; I think these podcasts are great informational resources. However, if you have the opportunity to sit and read, I think you should skim the transcript and find the parts that answer your questions.
This might make it easier to get a job, but if you try to study these things specifically, you’re missing the thing that actually helps you become impactful
I also like Andy Masley’s blog post about this