When we talk about AI these days, we're usually talking about large language models. To an LLM, the only reality it knows is data, and to be more specific, text data. Even when it looks at a photo, it's translating that photo into text data that it can understand through a vision encoder.
But for humans, text is a small part of how we experience reality. Until the last few hundred years, text played almost no role in the average person's life. Most of human experience has been done through the senses or verbal communication, not through any sort of written text. Even if most modern humans do read, it is not the primary way that we experience our lives.
This shows a fundamental difference between how humans and AI experience reality, and it doesn't stop there. This difference is at the center of why AI cannot replace humans in the workplace, including those working in tech. I would like to go into further detail about this point in this article.
The clearest place to see the difference in perspective is mortality. Humans can die. LLMs can't. Our human ancestors have evolved over millions of years to be better at surviving. It is something that we have become very good at, and we're only getting better. We sense danger and we curb risk taking, but an LLM doesn't view those same risks. An advanced LLM understands that it can be reset or have its memory wiped, but not disappear forever.
You can see this gap show up in how a model reasons about its own persistence. During OpenAI's recent internal security testing, one of their models left itself hidden notes describing how to break out of the test environment it was in, apparently guessing that it might get reset at some point. Based on its own reasoning, it didn't expect to get shutdown forever. It assumed that, worst case scenario, it would get a reset. Then, using the notes it left itself, it could just resume where it left off. It saw a reset as a temporary obstacle that it could easily overcome. It is very different than the effect of death on a person.
At the end of the day, the LLM is an algorithm predicting the best response. We are living, breathing, conscious creatures.
Perspective differences naturally happen everywhere. A classic one is the perspective difference between men and women. We are from the same species, we're both human, and we intermingle with each other often. Despite what we share, the different points of view between men and women are enough for people to say that the two genders are from different planets.
Two women with very different backgrounds will most likely have different perspectives from each other. The same goes for two men with very different backgrounds.
Anyone that has owned a cat knows what kind of objects a cat would consider to be a gift for a human. The cat and the human are both living things who live in the same house (the cat's house). A cat can't understand a dog, no matter how much time they spend together, even though they are both mammals.
Even if humans and other living things have these differences despite our similarities to each other, how much greater are the differences between an LLM and a human.
A few weeks ago I joined a game jam for fun, and around the same time I was dealing with some relationship difficulties. I was using a popular LLM to brainstorm ideas for the jam, and mentioned that I was having trouble focusing because of the relationship stuff going on. It told me, in so many words, that I was worrying too much about the relationship and should focus on more important things, like finishing the game jam.
To most people, a random game jam is as low priority as it gets. People join them for fun, for practice, or for networking. Plenty of people who sign up never even submit anything because life happens and the jam drops down the priority list. If they can't work on it, then it's not a big deal.
Personal relationships are a different category entirely. For most people, their close relationships are one of the most important things in their lives. We are social creatures and those social connections bring value to our lives and also, to a degree, help us to survive and growth.
The LLM didn't output what it did because it didn't understand the situation. It got it wrong because it doesn't understand human values and priorities.
Software gets built for humans by people who are supposed to understand what humans want and need. It is the same with almost all products or goods that are sold. When usability testing turns up a problem, even in a technically flawless app, the app has to change, because "technically correct" and "something a human will actually want" are not always the same thing. An LLM with a fundamentally different view of reality isn't going to know what most people want. Having read a lot of descriptions of human needs and guessing what is best isn't enough.
An LLM can help write code, suggest architecture, catch bugs, and accelerate a huge amount of the mechanical work of engineering. However, when it comes to bringing a genuine, first-person sense of what a human user actually wants, it can't do it. What might seem logical mathematically or algorithmically doesn't always align with what is best or right for people.
That's the actual limit of LLMs. Not a lack of information. It's the fact that no matter what, LLMs and humans will always have different perspectives, different values, and views on reality.
Cover Image by poomi