Reality Doesn’t Fit in a Prompt

A developer argues that large language models (LLMs) are fundamentally limited by their reliance on human-compressed prompts, which cannot capture the infinite context and lived experience that drives human judgment. The developer notes that while agentic systems and direct perception narrow this gap, the bottleneck remains: models lack the shared context and priorities that define human decision-making.

LLMs took the tech industry by storm and changed our relationship with machines. They can answer questions, reason through unfamiliar problems, and increasingly act on our behalf. Yet something still feels absent. A model can process a description of what is, but it does not share the lived context from which we decide what should be. That gap is what I want to explore. In 2022, ChatGPT launched and went viral. Everyone was amazed and excited. Yet by mid-2024, what a pure LLM could do had started to plateau. Then reasoning arrived, and the ceiling moved again. Models stopped merely answering and became genuinely capable of acting on your behalf. Agentic systems followed. Now the ambition is to let models perceive and act in the world more directly. Notice the pattern: every leap came from giving the model more room to see and act. And yet, as amazing as they are, LLMs still don't feel like Artificial General Intelligence AGI . Evolution made us state machines of a kind. Every waking moment, you are observing "What is" and deciding to act on "What should be". You are doing it right now, reading this. The mechanism is so sophisticated that we cannot comprehend it, and it is running you regardless. A way of evolution to navigate an infinite world. The two loops run in parallel. "What is" continuously consumes memory and sensory information. "What should be" continuously produces actions. Walk into a room and before you have formed a single word, you have already read the situation and know what needs doing. That takes milliseconds and fires an incomprehensible number of neurons. This is also how we model agents - only far more primitively. Nature had a few billion years of a head start. There is a huge gap in complexity between how humans operate and how LLMs do. Pick the simplest object near you and try to describe it completely. Not usefully - completely. You will find that every sentence you write demands three more. What it is made of, why it is here, what it is not, what you would notice if it changed. Description never bottoms out. A single chair is already infinite. And you hold thousands of these at once, effortlessly, without a word. That is what "What is" actually means. So the real limit is not the size of the context window. Even at a billion tokens, an infinite state has to squeeze through the narrow straw of what you can type. That is the bottleneck. So we built a workaround. The chat box is not really an interface, it is a compressor. You hold the infinite state, you decide what matters, you type a few sentences, and the model acts on that thin slice. It works remarkably well. Well enough that we stopped noticing how strange it is: the most capable software ever built, waiting for a human to tell it what the situation is. But the compression is lossier than it feels. Depending on the state you are in, the same words mean different things to you - and you are often not aware of which state that is. So the summary you type is not just incomplete, it is unstable. You cannot transmit what you cannot see in yourself. That is a partnership, not an intelligence. And notice who does the hard part. That bottleneck is already narrowing. Agents can search documents, observe screens, perceive their surroundings, and act through tools. The more they can perceive directly, the less they depend on us to compress reality into a prompt. But access to information is not the same as judgment. A model may see the same artifacts you see without knowing which constraints matter, which risks are acceptable, or which outcome is worth pursuing. You have had this moment. You describe a situation, the model responds with something confidently wrong, and you think: why don't you get it? Often, the problem is not its reasoning but the context and priorities it does not share. Even a perfect prompt cannot encode everything you know, because much of what you know was never put into words. This is what human judgment actually does, and it is easy to miss because it feels effortless. You absorb knowledge across a lifetime, recall the fragment that matters, and correct yourself through feedback you barely register. Then you decide what should be done. Current models can increasingly access the same information, but information alone does not determine what matters. But be honest about the other side. When a model has enough context and a bounded set of actions, it is already excellent. At RAW Labs https://raw-labs.com we've developed Kora https://kora.raw-labs.com : not to make an agent understand the entire world, but to define a world small enough for it to act reliably. Humans provide the context, constraints, and purpose; the model operates within them. Kora does not remove the narrow straw. It engineers it. One clarification: I deliberately wrote LLM and not AI. AI is a broader concept, and something beyond today's models could surpass these limitations. Nobody can predict that, and anyone who claims otherwise is selling something. Models will perceive more, and the boundary between human and machine judgment will continue to move. But the central problem remains: intelligence is not only processing what is. It is deciding what matters and what should happen next. For now, humans close that loop - either directly or by designing the context in which machines can act. Sources AI Engineer - Daniel Han talk https://www.youtube.com/watch?v=uIiA6DquRiE