A longer version of this article was first published in the Spring 2026 edition of 2600: The Hacker Quarterly under the title "Can AI Think?".
We Can't Prove Humans Think, So Why Need Proof AI Thinks?
There is currently no empirical evidence proving AI does or could one day think, and as to non-empirical evidence, no arguments have been made in the way all such scientific arguments are made. That is, no one has said “Here’s the principles of the system. Here’s the conclusion: ‘machines think’. And here’s the chain of reasoning showing how we progressed at each step from the principles to the conclusion”.
The reason no such arguments have been made may be that they cannot be made.
One problem is that experiences (for example) cannot be described in the language of computer programs, which we might call the language of mathematics or physics. I mean equations or propositions like 2 + 3 = 5 or such statements as “if billiard ball A strikes ball B, ball B will move”.
With such statements, you cannot get to experiences e.g. seeing the colour red. You can describe a chain of physical events—such as light waves entering the eye, signals moving through the brain, etc. But at the end of your description it would make just as much sense logically to put the experience of the colour blue or yellow as the colour red, or the experience of smelling the scent of a rose or of feeling pain.
There is a break in the chain of reasoning at the point at which experienced is introduced.
Since there are necessarily no unknowns in computer programs, we are able to relatively easily form hypotheses about what they are capable of.
For example, we could propose the following: “If you were able to tally up the number of times certain words followed other certain words in a large amount of text, you could produce coherent sentences by choosing high-frequency word combinations” (that’s what AI does). That’s a comprehensible hypothesis with connections between principles and conclusions.
But when it comes to machine intelligence, [no one has put](https://garymarcus.substack.com/p/the-five-stages-of-agi-grief) forward [a hypothesis](https://rodneybrooks.com/ai-great-expectations/).
Some people might argue that, in line with my reasoning, I have no grounds to say that humans (other than myself) think. I cannot look into either people or machines and identify “thinking” or “consciousness”. All I have to go off is their behaviour.
My answer is: there is a difference in the sorts of logical arguments we can formulate when it comes to the operations of machines vs the operations of the human mind—one that I’ve already pointed out above. That is, we know how machines function.
Computer programs are a series of explicit, known logical statements—sentences written down that if you opened the right documents you could read off your screen. Therefore, if you’re going to argue machines can think, you should be able to point to the sentences in the computer program that constitute the operations behind thinking. The analogous “formulations” (if we could use that word) behind the operations of the human brain, on the other hand, are not known, which is why I don’t hold the same standards when it comes to proving humans think.
.
.
.
Take a look at my other articles on Substack ('Oscar Davies on Artificial Intelligence'): On Ilya Sutskever’s failure to answer the Q: How will AGI be invented?
On Eliezer Yudkowsky’s unsubstantiated arguments
On what words mean to computers
.
__Twitter/X: __x.com/OscarMDavies