We (Parker and Tom) recently coauthored a paper, “The Economics of Recursive Self-Improvement”, with 7 other economists. The paper walks through a series of simple models of how AI may accelerate AI R&D, and we thought it’s worth highlighting some context and takeaways:
We care about Recursive Self-Improvement (RSI) because we want to forecast capabilities. METR’s priority is to assess risk from frontier AI development, and one input is how capable AI systems will be in the future. Capabilities have been growing rapidly over the past 5 years, and we want to know whether to expect an acceleration.1
The term RSI has been used with very different definitions. Unfortunately a lot of confusion has been caused by different definitions of RSI. Everyone agrees that RSI refers to feedback from model capabilities to model improvements, but some have said that RSI occurs when there’s any feedback (Karpathy, Patel, Musk, LessWrong), while others reserve it for when the feedback is strong enough to cause super-exponential growth (Lambert) or fully autonomous growth (Favaro & Clark). We decided not to use the term RSI in a technical sense, to avoid confusion. Instead we focus on the strength of feedback effects, and whether they are sufficiently strong for “self-sustaining acceleration.” (We have a longer survey of definitions here).
The effect on capabilities acceleration depends on the strength of feedback effects. The model gives a simple way of quantifying the strength of overall feedback effects through decomposing into individual effects. The most uncertain relationship is how an increase in model capabilities would increase the rate of algorithmic progress.
We can’t rule out a substantial acceleration. We discuss a variety of reasons why there could be an acceleration in capabilities that fizzles out: bottlenecks on data, training compute, inference compute, or experiments; algorithmic-specific capabilities; and R&D-specific capabilities. However, we do not think the evidence for any of these is overwhelming; we cannot rule out an extended and rapid acceleration in capabilities.
There is more data relevant to RSI that the labs could be releasing. Over the past 6 months labs have released a lot of useful data about the impact of AI on AI R&D (Mythos model card; GPT-5.6 model card; Favaro & Clark), but there are many more facts they could release that would be useful. The paper gives one specific “wish list” for future releases.
What next? The paper has a calibration, suggesting estimates for parameters, but it is very loose and meant to be a first draft. We hope to keep iterating on our quantitative model to give a more operationally useful model of RSI.
We are also interested in reasons why capabilities might decelerate, e.g. our paper on a slowdown in the growth of training compute. ↩