The Fatal AGI Hardware Gap

Hardware restrictions proposed to prevent dangerous superintelligent AI, such as limiting fabrication to 28 nanometer process nodes or computing capacity below 15,840 TFLOP/s, may inadvertently guarantee the creation of unethical AGI. A newly identified "fatal gap" exists between the minimum hardware required for any AGI and the minimum hardware needed for an ethical AGI, meaning hardware limits set within that gap would produce only unethical, potentially catastrophic systems.

Restricting computer hardware has been proposed as a solution to preventing the dangers of building superintelligent AI. For example, MIRI suggests restricting hardware fabricated on 28 nanometer process node processes or smaller or hardware with 15,840 TFLOP/s or greater computing capacity 1 https://www.lesswrong.com/feed.xml fn-9BXtGyiuum3ySExtA-1 There is a catch with restricting hardware however, that I have not seen discussed elsewhere. Essentially, there is a minimum level of hardware required to create an AGI which we can call . If the hardware is below this level which will actually be a frontier depending on multiple things like memory, compute speed and available I/O , the hardware will be incapable of AGI, if it is below the limit. If it is above, the hardware is capable of AGI. Now, let us suppose that the AGI is ethical, in that it can correctly choose actions that do not harm other beings in the universe. We can list another limit, for the minimum level of hardware required to create an ethical AGI. Now compare them. Imagine we have a minimum ethical AGI. We can make the AGI smaller and simpler by deleting things like how to figure out which atoms are currently being used by other beings, and instead just grab whatever atoms we can. So it seems quite likely that . 2 This means that there is a gap between the two, and if a hardware limit is set in this gap, the probability that an AGI is unethical is 100% In short, if we try and prevent AGI by using hardware limits, we need to make sure we are not in the gap between and because that would actually move into a region where any AGI is always fatal. 4 Above $L {EAGI} the probability of an AGI being unethical is less, but I am not sure what direction the probability goes with increasing computational power increases, decreases, or remains constant seem possible to me . seems impossible, since an ethical AGI is an AGI. might be possible if all or at least the lowest resource using AGI's figure out ethics more or less automatically. might also be possible if ethics are easy to add an existing AGI. ↩︎ https://www.lesswrong.com/feed.xml fnref-9BXtGyiuum3ySExtA-2 At least if the hardware limit is completely enforced. ↩︎ https://www.lesswrong.com/feed.xml fnref-9BXtGyiuum3ySExtA-3 Note that the MIRI limits seem more likely to be set to prevent superintelligence, rather than AGI. Eliezer Yudkowsky, Steve Byrnes, and myself have all estimated that current high end personal computers or lower probably can do AGI: https://intelligence.org/2022/03/01/ngo-and-yudkowsky-on-scientific-reasoning-and-pivotal-acts/ https://intelligence.org/2022/03/01/ngo-and-yudkowsky-on-scientific-reasoning-and-pivotal-acts/ , https://www.lesswrong.com/posts/LY7rovMiJ4FhHxmH5/thoughts-on-hardware-compute-requirements-for-agi https://www.lesswrong.com/posts/LY7rovMiJ4FhHxmH5/thoughts-on-hardware-compute-requirements-for-agi , https://www.researchgate.net/publication/388398902 Memory and FLOPS Hardware limits to Prevent AGI https://www.researchgate.net/publication/388398902 Memory and FLOPS Hardware limits to Prevent AGI ↩︎ https://www.lesswrong.com/feed.xml fnref-9BXtGyiuum3ySExtA-4