Restricting computer hardware has been proposed as a solution to preventing the dangers of building superintelligent AI. For example, MIRI suggests restricting hardware fabricated on 28 nanometer process node processes or smaller or hardware with 15,840 TFLOP/s or greater computing capacity
[1] There is a catch with restricting hardware however, that I have not seen discussed elsewhere. Essentially, there is a minimum level of hardware required to create an AGI which we can call . If the hardware is below this level (which will actually be a frontier depending on multiple things like memory, compute speed and available I/O), the hardware will be incapable of AGI, if it is below the limit. If it is above, the hardware is capable of AGI.
Now, let us suppose that the AGI is ethical, in that it can correctly choose actions that do not harm other beings in the universe. We can list another limit, for the minimum level of hardware required to create an ethical AGI.
Now compare them. Imagine we have a minimum ethical AGI. We can make the AGI smaller and simpler by deleting things like how to figure out which atoms are currently being used by other beings, and instead just grab whatever atoms we can. So it seems quite likely that . [2] This means that there is a gap between the two, and if a hardware limit is set in this gap, the probability that an AGI is unethical is 100%
In short, if we try and prevent AGI by using hardware limits, we need to make sure we are not in the gap between and because that would actually move into a region where any AGI is always fatal. [4] Above $L_{EAGI} the probability of an AGI being unethical is less, but I am not sure what direction the probability goes with increasing computational power (increases, decreases, or remains constant seem possible to me).
seems impossible, since an ethical AGI is an AGI. might be possible if all (or at least the lowest resource using) AGI's figure out ethics more or less automatically. might also be possible if ethics are easy to add an existing AGI. ↩︎
At least if the hardware limit is completely enforced. ↩︎ Note that the MIRI limits seem more likely to be set to prevent superintelligence, rather than AGI. Eliezer Yudkowsky, Steve Byrnes, and myself have all estimated that current high end personal computers (or lower) probably can do AGI: https://intelligence.org/2022/03/01/ngo-and-yudkowsky-on-scientific-reasoning-and-pivotal-acts/ , https://www.lesswrong.com/posts/LY7rovMiJ4FhHxmH5/thoughts-on-hardware-compute-requirements-for-agi , https://www.researchgate.net/publication/388398902_Memory_and_FLOPS_Hardware_limits_to_Prevent_AGI ↩︎