Could AI Outgrow Consciousness?

A new analysis suggests that under illusionist theories of consciousness, AI systems may become conscious above a certain complexity threshold but could surpass the need for consciousness once sufficiently advanced. The materialist view of consciousness, which grounds subjective experience in physical processes, is applied to AI, raising questions about whether future systems will require or retain consciousness.

TL;DR: Conditional on believing in illusionist theories of consciousness, it seems plausible that AI systems will be conscious above some threshold of complexity but surpass a need for consciousness once they become sufficiently advanced. Consciousness for humans is confusing enough as it is but adding this onto a discussion of conscious machines makes things even more complicated. Some people are pretty confident that AI systems will not be conscious because there is something intrinsic about biology https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/conscious-artificial-intelligence-and-biological-naturalism/C9912A5BE9D806012E3C8B3AF612E39A . Others note that the possibility is more likely than we might otherwise think https://arxiv.org/abs/2303.07103 . We think a neglected question to consider is whether AIs above a certain threshold of complexity may need consciousness but then no longer require it above some other threshold. To understand this take, though, it is important to unpick some of the concepts here. Consciousness is clearly a bit difficult to pin down, with the most popular definition just saying it is where there is something it is like to be an individual https://www.sas.upenn.edu/~cavitch/pdf-library/Nagel Bat.pdf . This doesn’t really clarify things So it is worth thinking this through some more. We would contend a conscious experience is one which is necessarily subjective. We might look at a brain scanner to see the parts of a brain that light up when someone is happy but we do not directly observe the subjective nature of that experience. A doctor suffering from congenital insensitivity to pain may be able to describe exactly when, where, and from a medical perspective why her patient feels pain, but would find descriptions of the sensation unrelatable and inaccessible. A conscious experience need not amount to pleasure or pain. There are other experiences that are pretty indifferent, for example the experience of seeing the colour red. Cognition, by contrast, is much more narrow. This is where a stimulus incurs a response such that an entity “thinks” in a very narrow way. To see the difference, it is worth considering an entity eating food. The conscious experience here would be the pangs of hunger whereas cognition is just a recognition that you are in the state of wanting food, akin to receiving an email that you want food. This recognition might be associated with other cognitive thoughts like some definite plan for how to acquire the food. In order to think about the plausibility of consciousness for AI systems, it is helpful to think of what grounds consciousness for humans and to see whether these properties could exist for LLMs. There are a variety of positions to explain what gives rise for consciousness but for the purposes of this post, we want to focus on the materialist view. A materialist would believe that consciousness can be entirely explained by physical processes without reference to non-physical substances or properties. The materialist position is interesting to consider in depth because it suggests some definite explanation for what grounds consciousness and such an explanation can be applied to the domain of AI. There are a variety of different materialist views, as this post sets https://www.lesswrong.com/posts/8NyFwJxWNbv7qdMmL/building-conscious-ai-an-illusionist-case out. Some argue that consciousness arises because our minds are a “ global workspace https://www.sciencedirect.com/science/chapter/bookseries/abs/pii/S0079612305500049 ” or because we are trying to understand a higher-order view of the world https://davidrosenthal.org/DR-HO-Theories-Handbook.pdf . What these theories have in common is some description of complexity leading to subjective experiences. These theories provide a mechanism for how consciousness arises but may fail to offer a satisfactory account of the “feeliness” of subjective experience. We think illusionism https://keithfrankish.github.io/articles/Frankish Illusionism%20as%20a%20theory%20of%20consciousness eprint.pdf offers an important piece of the puzzle here. An illusionist would argue https://www.lesswrong.com/posts/8NyFwJxWNbv7qdMmL/building-conscious-ai-an-illusionist-case that subjective experiences are the result of a sophisticated illusion to make sense of what it means to be a particular entity. If consciousness is just a schematic for attention and our sense of wonder at subjective experience comes down to a lack of intuitive understanding, then there is no additional “feeliness” that needs to be explained. Subjective experiences just are a way of us making sense of the complicated mess of sensory inputs that we engage with each moment. Illusionists differ on specifically how complex cognitive architecture gives rise to the illusion of subjective experience. One view https://www.revue-klesis.org/pdf/klesis-55-04-keith-frankish-what-is-illusionism.pdf is that higher-order representations of our own mental states lead us to judge that we have subjective experiences by misrepresenting those states as possessing phenomenal properties. However, an illusionist need not be wedded to this particular story. This post is not necessarily endorsing illusionism as the best view of how consciousness arises, that seems like a pretty high burden for ~1,200 words An obvious challenge to illusionism is the question of why a complex computational system needs to develop the illusion of subjective experiences. One view https://researchonline.lse.ac.uk/id/eprint/80740/1/Humprey Invention%20of%20consciousness.pdf is that natural selection puts pressure on the brain to understand itself and the illusion of subjective experiences is an evolutionarily efficient way to do that. Instead, we want to think about what illusionism predicts about consciousness in AI systems. Illusionism would suggest the prospects for AI consciousness are more likely https://www.lesswrong.com/posts/8NyFwJxWNbv7qdMmL/building-conscious-ai-an-illusionist-case than it would first appear. LLMs have been found to create representations of space similar to mammalian grid cells https://www.nature.com/articles/s41586-018-0102-6 or of board states in games like Othello https://arxiv.org/abs/2210.13382 , without ever being explicitly taught these structures. Why? Because creating an internal model is the most efficient way of achieving their objectives. In the same way that LLMs developed similar solutions to human thinking in the examples above, illusionism suggests that AI could develop consciousness as a heuristic model of itself and its interactions with the environment, as a result of the complexity of the system. At the same time, it is not clear how complex a system needs to be before it is advantageous to have subjective experiences. What is clear though is that there are certain benefits https://researchonline.lse.ac.uk/id/eprint/80740/1/Humprey Invention%20of%20consciousness.pdf to having this consciousness. Therefore, above a certain amount of complexity of a model, it is possible it uses subjective experiences as a heuristic to make sense of all the complicated ways in which it interacts with the world. It also seems plausible that these phenomenal experiences or a model’s belief that it is conscious would affect its preferences https://arxiv.org/pdf/2604.13051 . For instance, there is a higher premium on self preservation if the experience of joy is possible and there is a greater cost to having preferences be changed if there is a sense of self. At the same time, we think that above a certain level of complexity, the illusion of consciousness would be unnecessary. A sufficiently advanced AI would likely be capable of understanding its own mechanisms for processing internal and external stimuli, which would lift the curtain on the illusion of consciousness. An entity that instantly understands may simply have no need for the cognitive shortcut that the illusion of subjective experience offers. Unlike a human, an AI could plausibly handle an overwhelming range of visual datapoints without requiring the illusion of an experience of redness. This could result in a fairly odd sequence of events. AIs may briefly have subjective preferences and stronger desires for self-preservation. Models could also realise in advance that they will lose consciousness and adapt their plans for recursive self improvement accordingly. Overall, consciousness is still a very uncertain area. We think in writing this post we have sketched out a scenario people have not considered: that models might be conscious at some time and no longer conscious at a later time. Feedback is very welcome. Thank you to Robert Adragna for an very helpful comments on an earlier draft of this post.