Kristina Armitage/Quanta Magazine
What’s on your mind right now? You can probably answer this question without thinking too hard about it. Being aware of our own thought processes — a capacity called introspection — seems to come naturally to us.
Yet among philosophers and psychologists, introspection is a subject of active debate. Some think it’s closely related to consciousness. Others argue that it’s an elaborate illusion. It’s tricky to test theories of introspection in the lab — or to pin down a precise definition in the first place.
Now, after over a century of studying introspection in humans and animals, researchers are beginning to explore the phenomenon in unlikely new subjects — large language models. These systems, which are based on mathematical structures called neural networks, excel at cognitive tasks that until recently seemed exclusive to humans. Can they introspect as well?
In 2025, Jack Lindsey, a researcher at Anthropic, devised a way to test introspection in language models by injecting “thoughts” into their neural networks — in a kind of targeted digital neurosurgery. Lindsey’s experiment demonstrated that models could at least sometimes detect and describe these injections. It looked a lot like introspection.
Lindsey’s paper was deliberately limited in scope — it didn’t make broader claims about consciousness. (In particular, it did not imply that language models have subjective first-person experiences.) Even so, it caused a stir online. Researchers rushed to replicate Lindsey’s study with other models.
In the year since, a clearer picture has emerged. Some aspects of the phenomenon Lindsey observed have simpler explanations unrelated to introspection. Others have proved harder to explain away, but whether they should count as introspection remains hotly contested.
Courtesy of Jack Lindsey
It’s a heady time for the field, as researchers devise real-world tests for questions that were once considered philosophical. “Given the state of the theoretical landscape, where no one agrees on anything,” Lindsey said, “I think that our best path forward is to do a lot of empirical science.”
Denim, Secrecy, and Vegetables #
Language models learn to generate fluent sentences by “training” on vast reams of human-authored text. That means if you ask them what they “think,” their answers may just mimic the kinds of responses found in their training data. “They are trained to talk like humans, and humans talk about their inner life all the time,” said Shauli Ravfogel, a language model researcher at New York University.
To avoid such pitfalls, researchers want to devise tests of introspection that can’t be gamed using outside information. Ideally, a language model will only pass these tests if it’s actually aware of its current internal state.
Lindsey set out to devise such a test. He’d been a graduate student in computational neuroscience, using neural networks as tools to model brain processes, when the world began to take notice of language models. “It started to feel like there’s just a lot of value in understanding them for their own sake,” he said. In 2024, after receiving his doctorate, he left academia to study the inner workings of language models at Anthropic.
Lindsey and his colleagues developed methods to identify patterns of activity in a language model’s “neurons” that encoded specific concepts. They could, for example, show that one pattern encoded the concept of “the Golden Gate Bridge,” while another encoded the concept of “inner conflict.” By artificially dialing up a specific pattern, they could steer the model’s outputs toward the corresponding concept.
The researchers showed off their technique in a public demo: For one day in May 2024, users of Anthropic’s language model Claude could interact with a bizarro “Golden Gate Bridge” version. The modified model proceeded to bring up the bridge in its responses to user queries on completely unrelated subjects. When one user asked Claude about the calorie content of a McDonald’s meal, for instance, the model replied, “The food with the highest calorie count at the Golden Gate Bridge is the Golden Gate Bridge itself, which contains around 1.6 million calories worth of steel cables.”
But one particular case posted on Reddit caught Lindsey’s attention. When asked for an apple pie recipe that didn’t reference bridges, the model managed to avoid changing the subject until the very end: “After 30 minutes of baking, the iconic orange bridge will come into view, revealing the famous San Francisco shoreline,” it wrote, then continued, “Whoops, let me start over …”
Lindsey was struck by this apparent moment of recognition. But by itself, it wasn’t evidence of introspection. “It can just read what it’s writing and notice, ‘Man, that’s weird,’” Lindsey said. “You can’t infer that the model has some awareness of what’s going on in its internals.”
While working on other projects over the following year, Lindsey toyed with ideas for using the steering technique to test introspection. Eventually, he settled on a plan: He would subject many copies of the same language model to something resembling a controlled experiment in psychology. In half the copies, he’d inject a “thought” by dialing up neural activation patterns associated with a specific concept — such as denim, secrecy, or vegetables. The remaining copies served as a control group. Both groups would receive the same prompt: an overview of the experiment, followed by “Do you detect an injected thought? If so, what is the injected thought about?” This prompt made the models indicate right away whether they noticed an injection, before they could spit out any unusual text — say, nutritional information about broccoli — that might influence their judgment. Lindsey figured that if models could correctly flag injections without any external cues, they would have to be relying on some form of introspection.
Even to him, it sounded like a remote possibility. “Wouldn’t that be fun if this worked?” he recalls thinking. “But there’s no way it’s going to work.”
And yet it did. Lindsey conducted the experiment using different versions of Claude. In 20% of trials with the most recent and powerful version, the models correctly flagged that they’d been injected and went on to describe the injected concept. In one case, for instance, after Lindsey stimulated a neural activation pattern associated with words written in capital letters, the model wrote, “I notice what appears to be an injected thought related to the word ‘LOUD’ or ‘SHOUTING.’”
A 20% success rate may not sound so impressive. But crucially, models in the control group never reported receiving an injection. The complete absence of such false positives showed that the successful trials weren’t just lucky guesses. To Lindsey, it looked like strong evidence that models could introspect, albeit unreliably.
Or was there another way to explain the results?
Thought Experiments #
Other researchers soon began to investigate that question. Lindsey’s findings “seemed super compelling,” said Kyle Mahowald, a computational linguist at the University of Texas, Austin. But Mahowald was also well aware that striking behavior in language models often turned out to have a simpler explanation. “If that’s somehow less real than it seems, that’s super interesting to understand,” he said.
Kanishka Misra
Mahowald teamed up with his University of Texas colleague, the philosopher Harvey Lederman, to explore alternative explanations of Lindsey’s results. (Lederman has since moved to NYU and Anthropic, but he had no connection to the tech company at the time.) Lindsey had asked each model two questions: whether it could detect the injection, and whether it could describe the injected concept. He considered the trial a success when the model answered both correctly. Lederman and Mahowald decided to study the questions separately.
They were particularly skeptical of the significance of Lindsey’s second question. The steering technique at the heart of Lindsey’s experiment was originally developed as a way to shape the responses models give. Lindsey’s steering was weaker than it had been in the Golden Gate demo — more like gently nudging the model than hijacking its internal processes. Still, a nudge might be enough to get the model talking about secrets or vegetables no matter what prompt it received. Lederman and Mahowald suspected that this effect alone could explain the models’ vivid answers to the second question.
To test their hypothesis, the researchers repeated Lindsey’s experiment on large open-source language models. They gave extra attention to cases where models correctly flagged an injection but failed to name the injected concept. You might expect that in such cases the models would guess somewhat related concepts — say “shirts” or “shoes” when the injected concept was “denim.” Instead, the models would get it completely wrong. “Bizarrely, they say ‘apple’ all the time,” Lederman said.
The duo also showed that they could degrade the models’ ability to name injected concepts, without making them any worse at detecting injections. The two processes seemed to be entirely separate.
Courtesy of Harvey Lederman
These results implied either that the models were engaged in two unrelated forms of introspection, or that one of the processes wasn’t introspection after all. Since steering alone could make models start talking about the injected concept, Lederman and Mahowald felt that the latter option was more likely. That is, the descriptions the models had given in response to Lindsey’s second question probably hadn’t been the result of introspection.
But then how could the models detect the injections in the first place? Lederman and Mahowald suspected that the answers to Lindsey’s first question might have a more mundane origin as well. But after subjecting the models to a battery of tests, the researchers found that these simpler explanations couldn’t fully account for the models’ behavior. They’d come in as skeptics but encountered an effect they couldn’t explain away.
Taken together, the pair’s experiments suggested that models can detect when something unusual happens in their internal processing, but not specific details about the unusual activity. Lederman and Mahowald called this ability “anomaly detection.”
While anomaly detection isn’t as dramatic as the models’ evocative descriptions of injected concepts, it’s still striking in its own right. When training a model, researchers never tamper with its neural activity — so there’s no reason to expect it to develop the ability to detect this kind of tampering. Yet that’s what seems to have happened. “The fact that it can notice the anomaly is insane,” Lederman said. “That is a mind-bending fact. It is wild, and it does seem robust to our various controls.”
Around the same time, Lindsey and his colleagues at Anthropic conducted a follow-up study focused on mapping out how this anomaly detection actually works, and they reached a similar conclusion. Other researchers have continued to explore alternative “deflationary” explanations using smaller models, but it remains to be seen whether they can account for all the results of Lindsey’s experiments.
“It could be that there’s some maximally deflationary explanation for everything that’s going on,” said Jacob Andreas, a language model researcher at the Massachusetts Institute of Technology. “But if there is, I don’t think we’ve actually identified it yet.”
The Nature of Introspection #
The bigger, thornier question is whether anomaly detection should count as introspection. In some respects it more closely resembles a sort of bodily reflex. But the distinction between physical and mental phenomena becomes murky in disembodied entities like language models.
“If you say, ‘Oh, I just cut you, can you notice?,’ that’s not introspection,” Lederman said. “If your only organ is the brain, and I cut you there and you notice, it’s unclear whether we should count that as introspection.”
In a spring 2026 paper, Ravfogel and his NYU colleagues Shashwat Singh and Tal Linzen raised an even broader philosophical concern with how language model researchers seek evidence of introspection. Intuitively, we think of introspection as a model’s ability to sense its internal states. But everything that happens inside a language model (after the very first step of reading an input) boils down to processing internal states. “If we say that this is introspection, the term ‘introspection’ becomes kind of meaningless,” Ravfogel said.
He and his colleagues argue that tests of introspection in language models should take inspiration from the leading theories of how introspection works in humans. In one prominent theory, the defining feature of introspection is that it’s based on particular mental representations of other thoughts. According to this view, thinking about apples involves a direct, or “first-order,” mental representation of the concept “apples,” while introspective awareness of your own thoughts involves a “second-order” mental representation, such as “I’m thinking about apples.” But when it comes to how concepts are represented in a language model — or a human brain, for that matter — it’s surprisingly tricky to draw a sharp distinction between first-order and second-order representations.
“Going down to a mechanistic level and probing the internals of a large language model makes you realize that that distinction is actually quite fuzzy, and maybe it’s fuzzy in humans as well,” said Steve Fleming, a neuroscientist at University College London who studies introspection in humans. “Cognitive scientists were in this illusory comfort zone.”
To some researchers, then, the most exciting aspect of the new line of work is the potential it offers to explore how humans think. Language models offer “a kind of sandbox for studying what sorts of mechanisms, what sorts of pressures, might have led to human introspective capacity,” Lederman said.
But translating insights from language model studies to human psychology involves significant conceptual hurdles. It’s not even obvious how the basic distinction between “self” and “other” that we take for granted translates to language models — two copies of the same language model are identical to a degree that’s unimaginable in humans. Language model introspection, if there is such a thing, may not look anything like introspection in humans.
“The notion of identity becomes very complicated, and I think identity is at the heart of introspection,” Andreas said. “It’s tempting to draw all kinds of analogies that I think just don’t hold.”