Welcome to The Closing Argument: our verdict on the news, plus our writers’ key takeaways and everything The Argument published and appeared in this week.
The Verdict by Kelsey Piper #
The leading AI companies, Anthropic and OpenAI, have both said that they are focused on achieving recursive self-improvement (RSI). RSI is a possible next stage of AI development, in which humans turn over control of AI development to AIs themselves — and, by implication, making human oversight an increasingly marginal part of the process as AI grows faster and more powerful.
Many people working in AI, including respected senior scientists, think RSI could be a disaster for humanity. But the leaders of the AI labs tell us that they’re hard at work on this in part because, if they don’t do it, someone else will. “We focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward,” OpenAI Chief Scientist Jakub Pachocki wrote in a September essay.
This week, we got significant new information about the state of the race toward RSI. Well, sort of: We got a report from the third-party AI research nonprofit METR on this topic, but much of its contents are redacted at the request of Anthropic because they’re “commercially sensitive.”
But what we do know about the report is this: Anthropic says that it expects RSI very soon. METR estimates, with substantial uncertainty, that on well-specified tasks, progress that used to take a full year now happens in roughly six months. Anthropic and METR see no signs that anything will slow this process down. Crucially, this isn’t just a matter of spending more money, but by the increased ability of AI itself to hasten research.
But we have to take their word for it, because almost all the evidence of this in the report was redacted.