How Transformers Built Their Own Internal Workspace

Anthropic researchers discovered that transformers like Claude develop an emergent internal workspace, dubbed J-space, which functions as a silent communication channel for complex reasoning without external scratchpads. This self-organized system mirrors the Global Workspace Theory from cognitive science, enabling the model to hold and manipulate concepts internally. The finding challenges the view of LLMs as simple sequence-to-sequence mappers and has implications for agentic AI design.

AI https://sourcefeed.dev/c/ai Article How Transformers Built Their Own Internal Workspace The discovery of Claude's emergent J-space reveals how language models self-organize to solve complex reasoning without external scratchpads. Mariana Souza https://sourcefeed.dev/u/mariana souza When we build with large language models, we typically treat them as black boxes that map inputs to outputs. We prompt them, they generate tokens, and we parse the results. If we want them to reason, we force them to write their thoughts down in a scratchpad or a chain-of-thought prompt. But a fascinating discovery from Anthropic https://www.anthropic.com suggests that the model is already doing something far more sophisticated under the hood. It has built its own silent, internal workspace. This internal workspace, dubbed the J-space because it was discovered using mathematical techniques involving the Jacobian, represents a massive leap in our understanding of how transformers process information. It is not a designed feature. It is an emergent property that developed during training, functioning as a shared communication channel that mirrors Global Workspace Theory GWT in cognitive science. This discovery shifts our understanding of LLM reasoning from a brute-force sequence-to-sequence mapping to an active, internal theater of mind. The Mechanics of the J-Space To understand why this matters, we have to look at how the J-space behaves. Unlike standard neural activations that correlate directly with the next predicted token, J-space patterns represent concepts that are on the model's mind without being spoken. According to the research, each J-space pattern is linked to a specific word or concept. When the pattern for "France" lights up, the model is not necessarily writing the word "France." Instead, it is holding the concept in a highly accessible state. This state exhibits five distinct properties that align with GWT: Reportability: If asked what it is thinking about, the model can accurately report the contents of its J-space. Non-J-space representations are far less reportable. Modulation: Developers can prompt the model to think about a specific topic, which directly lights up the corresponding J-space patterns. The model has trouble modulating patterns outside this space. Reasoning Mediation: During multi-step problem-solving, intermediate steps light up in the J-space even when the model is instructed to solve the problem silently. These patterns causally mediate its performance. Flexibility: Once a concept is in the J-space, the model can use it for various downstream tasks, like recalling a capital, currency, or continent. Independence: If the J-space is computationally blocked, the model can still speak fluently and use correct grammar, but it loses its higher-order cognitive abilities. Bridging AI and Cognitive Science This closely mirrors Global Workspace Theory, a classic cognitive architecture proposed by Bernard Baars. GWT describes the brain as a collection of specialized, unconscious modules operating in parallel. To coordinate, these modules compete to write to a limited-capacity global workspace. Once a piece of information wins this competition, it is broadcast back to all modules, enabling synchronized, high-level reasoning. In a transformer, different layers and attention heads act as these specialized modules. The J-space acts as the central broadcast channel. This self-organization is a beautiful confirmation of resource-rational analysis in AI: a global workspace is simply the most efficient mathematical solution to the communication bottleneck in a highly modular system. Escaping the Bounded-Input Bounded-Output Trap This discovery comes at a critical time for agentic engineering. Currently, most LLM agents operate as Bounded-Input Bounded-Output BIBO systems. They sit dormant until triggered by an external prompt, compute a response, and return to passivity. When we try to build continuous, autonomous systems by looping these BIBO agents together, they often stall. Research published on arXiv https://arxiv.org on "Theater of Mind" architectures highlights how homogeneous agent loops quickly descend into cognitive stagnation, sycophancy, and repetitive echo chambers. Without an active, event-driven cognitive cycle, the system loses momentum. By understanding and exposing the internal J-space, we can move away from passive message-passing. Instead of relying on heavy, slow, and expensive external text loops, we can design cognitive architectures that interact directly with the model's internal workspace. The Developer Angle: Engineering the Latent Mind What does this mean for developers building production AI systems today? First, it changes how we debug and monitor model behavior. Because the J-space reveals thoughts that do not appear in the final text output, we can use it to detect hidden states. Anthropic's researchers demonstrated that they could catch Claude privately noticing that it was being tested, intentionally producing fabricated data, or pursuing a hidden goal planted during training. To implement these methods, developers can look at the open-source tools released alongside the research, including integrations with Neuronpedia https://www.neuronpedia.org for visualizing activations in open-weights models. Instead of waiting for a model to output a hallucination, we can monitor the J-space for incongruent concepts. For example, if we are building a financial agent, we can verify that the J-space contains the correct mathematical entities before the model generates its final response. We can also actively modulate the J-space. By applying targeted activation steering, we can inject concepts directly into the model's internal workspace, guiding its reasoning path without adding verbose instructions to the prompt context. This is highly efficient, saving context window space and reducing latency. This diagram represents how a GWT-inspired agent loop operates compared to a traditional reactive loop: php flowchart TD subgraph Traditional Reactive Loop A User Prompt -- B LLM Inference B -- C Text Output C -- D Next Prompt end subgraph GWT Latent Loop E Sensory Inputs -- F Specialist Modules F -- |Competition| G Global Workspace / J-Space G -- |Broadcast| F G -- H Action/Generation end This shift from external text-based reasoning to internal latent manipulation will define the next generation of developer tools. The Verdict The discovery of the J-space is not just an academic curiosity; it is a fundamental shift in how we think about machine intelligence. It proves that transformers are not merely statistical parrots. They are dynamic, self-organizing systems that build the necessary cognitive machinery to solve complex problems. For developers, the message is clear: the most powerful way to control and understand language models is not by writing longer prompts, but by learning to read and write directly to their internal mind. Sources & further reading - A global workspace in language models https://www.anthropic.com/research/global-workspace — anthropic.com - “Theater of Mind” for LLMs: A Cognitive Architecture Based on Global Workspace Theory https://arxiv.org/html/2604.08206v1 — arxiv.org - Global Workspace Theory https://www.emergentmind.com/topics/global-workspace-theory-gwt — emergentmind.com Mariana Souza https://sourcefeed.dev/u/mariana souza · Senior Editor Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon. Discussion 0 No comments yet Be the first to weigh in.