Humans Are Not Chinchillas: Revisiting “How Quick and Big Would A Software Intelligence Explosion Be?” Forethought researcher Tom Davidson's model of a software intelligence explosion, revised in a new Forethought article, finds it less likely that 10 or more years of total AI progress could fit into 1 year from software improvements alone, though 3 or more years of progress in 1 year remains somewhat likely. The revision lowers the takeoff ceiling and the parameter r, arguing that Chinchilla scaling by itself almost certainly does not allow a 4-5 order-of-magnitude increase in training efficiency past human efficiency. The author states his all-things-considered view is that 10 years of progress in 1 year is somewhat unlikely, while noting even a few years packed into one year could be disruptive and dangerous. This article was created by Forethought https://www.forethought.org/about . See all our research on our website https://www.forethought.org/research . Introduction Tom Davidson and Tom Houlden’s prior model https://www.forethought.org/research/how-quick-and-big-would-a-software-intelligence-explosion-be of an intelligence explosion poses the question: after AIs can entirely substitute for human researchers at the task of improving AI, how many year-equivalents of AI progress might be packed into a short period of time solely from further software progress, holding hardware constant? In this article, I build upon Davidson & Houlden’s model by reconsidering arguments for several parameter values. My chief contribution is discussing why some of the original arguments for more aggressive parameter values were weak: - “Chinchilla scaling,” by itself, almost certainly does not allow an increase of 4-5 orders of magnitude OOMs of training efficiency past human efficiency. - Other considerations also lower the likely ceiling of possible training efficiency. - Looking at evidence for the value of “r” from Epoch may lower its value as well. After adjusting parameter values accordingly, the model suggests that ≥3 years of total AI progress fitting into 1 year remains somewhat likely, but that it is less likely that ≥10 years of total AI progress will fit into 1 year. Note that, as the prior work remarks, any analysis of a software intelligence explosion SIE is necessarily speculative and involves guesswork and intuition. For full understanding of context, I recommend reading Davidson’s prior work. My all-things-considered view is that it’s somewhat unlikely for 10 years of total AI progress to fit into 1 year from software improvements alone; the output of this model, with my modified parameter values, is part of my reason for thinking this. I would nevertheless put higher odds on ≥10 years of progress in 1 year than does the model. My uncertainty over both parameter values and over the applicability of the model remains high, and, naturally, even a few https://www.lesswrong.com/posts/jfwhvd43sbpkGTLyn/full-automation-of-ai-r-and-d-probably-yields-a-large-speed years of total AI progress packed into one year might be quite disruptive and dangerous. Parameter Values & Model Output There are four central parameters to Davidson & Houlden’s model. I alter the distribution of two of these: r and the takeoff ceiling. The parameter r gives the initial returns to software R&D at the start of the SIE. Software progress tends to speed up over time if r is greater than 1, and slow down over time if r is less than one. See Eth & Davidson for more https://www.forethought.org/research/will-ai-r-and-d-automation-cause-a-software-intelligence-explosion article explanation. And the “takeoff ceiling” gives how many orders of magnitude of improvement in algorithmic progress are possible once complete AI substitution for human research has occurred and after the SIE has started. I modify the parameters as follows: After making these modifications, the likelihood of getting N years of total AI improvements within some specific amount of time changes as follows: So the likelihood of 3 years of progress over a short amount of time drops moderately, while the likelihood of 10 years of progress drops steeply. Overall, I reflectively endorse a significantly decreased likelihood of 10 years or more of total progress in 1 year or in 4 months. But I put less weight on the relatively small shift in 3 years of progress. In general – as I’ll discuss in the conclusion – it seems like a mistake to put too much weight on a model that is not specifically of the intelligence explosion, and is instead one which works through the proxy of more efficiency in parallel labor. This model does account for this through adjustment to the parameter values, but these adjustments are somewhat ad-hoc. Adjustments to the Takeoff Ceiling One of the key components of the original argument is that, after the first ASARA AI System for AI R&D Automation is trained, there will be between 6 and 16 OOMs of potential hardware-compatible, software-only algorithmic efficiency improvements still remaining to be found. That is, per the original discussion, suppose that the first ASARA system is trained with 10