{"slug": "j-space-yet-another-llm-mind-reader", "title": "J-Space: Yet Another LLM Mind Reader?", "summary": "Anthropic researchers have developed a new interpretability technique called the Jacobian lens (J-lens) to monitor internal representations in their Claude language model, revealing separate processing routes for automatic language use and explicit reasoning. While the findings have sparked interest among neuroscientists for resembling theories of access consciousness, the authors caution that the results do not demonstrate subjective experience or consciousness in the model.", "body_md": "Updated • 71\n\n# J-Space: Yet Another LLM Mind Reader?\n\n[Community Article](/blog/community)\n\n*Anthropic has managed to monitor Claude’s internal thoughts. Again? It feels as though we have heard that claim several times already. Ah, but this time the headlines say they found consciousness!*\n\nIf you're like me, you may feel that there have already been many papers and claims about \"reading the LLM mind\" in the past few years. Anthropic's earlier [ On the Biology of a Large Language Model](https://transformer-circuits.pub/2025/attribution-graphs/biology.html), for example, traced hidden multi-step reasoning, poetry planning and multilingual circuits in Claude 3.5 Haiku. What is different here? Is this consciousness business real? It has at least sparked interest among working neuroscientists, so it is worth a look. Let me explain what is genuinely new and exciting about this paper, and why the result is both more serious and less sensational than that opening sentence.\n\n## Do you speak Spanish?\n\nLet's start with a telling experiment. The researchers gave Claude a Spanish passage without naming the language and asked it to continue in the same language. In a second condition, they asked it to name a famous author who wrote in that language. The results are below on the left.\n\nNo surprises: the model is able to continue in Spanish and to name “García Márquez.” But the researchers then exchange its internal representation of *Spanish* for *French*. The results are on the right. Claude now answers “Victor Hugo” in the second case… but still continues the passage in perfectly good Spanish in the first one!\n\nThe model therefore seems to have at least two representational components or routes for using the same fact: one supports automatic language processing, while the other makes *Spanish* available for explicit report and flexible reasoning. The intervention disrupts only the second.\n\nCalling that second route “conscious” would be getting ahead of ourselves, but it resembles what philosopher Ned Block called [ access consciousness](https://www.nedblock.us/papers/1995_Function.pdf): information being made available for report, deliberate reasoning and flexible control.\n\nTo understand why, we need the technique the authors introduce: the [Jacobian lens, or J-lens](https://transformer-circuits.pub/2026/workspace/index.html). We will compare it with familiar interpretability tools, reproduce a small example on an open model, and examine why the representations it finds look like what neuroscientists call a “global workspace.”\n\nThen we can return to the unavoidable question: does any of this demonstrate that Claude has subjective experiences? Spoiler: no. It does not demonstrate the opposite either. But the actual result is interesting enough.\n\n## What would it mean to find a thought in a language model?\n\nThis attempt to reverse-engineer how models implement computations is the field of [ mechanistic interpretability](https://transformer-circuits.pub/2021/framework/index.html). Among other things, researchers try to identify how LLMs represent and manipulate concepts.\n\nThe naive picture of interpretability would be to identify a collection of neurons labeled *cat*, *joy* or *deception*, and perhaps show that when the “cat” neuron fires it triggers the “joy” neuron. Unfortunately, real neural networks are not organized so conveniently: a concept is usually distributed across many activations, while an individual neuron may participate in several apparently unrelated computations.\n\nIn many experiments, however, concept-related information behaves approximately like **a direction in the model’s activation space**. If you add or remove the right vector, you can strengthen, suppress or replace a concept. This picture is closely related to the [superposition hypothesis](https://transformer-circuits.pub/2022/toy_model/index.html), in which models represent more features than they have individual dimensions.\n\nSo when someone claims to have found an internal representation of a certain concept, we want at least three things:\n\n- A way to identify and interpret it.\n- Evidence that it has a causal role, rather than merely being correlated with the answer.\n- Evidence that the same representation is used across different contexts and operations.\n\nThe J-lens is interesting because the paper does more than produce evocative lists of internal words: it systematically tests all three ideas. But to understand that, we first need to look at how researchers usually look for concepts in an LLM's “thoughts.”\n\n## The transformer revisited\n\nIn the image below, you can see a familiar representation of the [transformer](https://arxiv.org/abs/1706.03762) on the left. Although there are many variants, the broad pattern is the same: token embeddings, then a stack of layers containing attention blocks (which move information between token positions) and MLP blocks (which process information within each position), and finally an unembedding, or output projection, that produces logits for the next token. Normalization operations are omitted for clarity. One apparently small but crucial detail is the *skip connection*, or *residual connection*, highlighted in red. It connects the input of a block to its output, so what is passed onward is the sum of the original input and the block's update.\n\nMechanistic-interpretability researchers often redraw the same wires to obtain the representation on the right. Look carefully: it is the same architecture from a different perspective. This time the residual stream is placed at the center, with every block in the stack connected to it.\n\nThis long central wire is called the **residual stream**, and it can be seen as a kind of memory or communication channel that the different blocks read from and write to. More precisely, there is one residual-stream vector at each token position. It starts with that token's embedding; attention moves information between positions, and each attention or MLP block writes an additive update. Finally, after all the layers, normalization and the unembedding turn the residual stream into logits, which softmax converts into token probabilities.\n\nThe residual stream is often treated as the model's main representation space: it records the model's evolving state as information is processed through the layers. For a given token position at a given layer, there is a vector of activations whose width is the model dimension, usually several thousand. This is where we will look for patterns representing concepts.\n\n## The logit lens\n\nThe simplest way to inspect an intermediate layer is the [ logit lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens). Take the residual-stream activation at layer and send it directly through the model’s final normalization and unembedding:\n\nThis is equivalent to asking which token would be produced if every transformer block after layer contributed no further update. We are roughly interrupting the model mid-thought, and we can scan the layers to see how its next-token prediction evolves with depth.\n\nTo visualize its effect, let's consider a typical task in mechanistic interpretability: two-hop reasoning. If I ask you, “What is the number of legs on the animal that spins webs?”, you have to go through an intermediate step: first recall that the animal spinning webs is a spider, and then remember that a spider has eight legs.\n\nHere is an example I created using the open-weights `google/gemma-3-4b-pt`\n\nmodel, which has 34 layers. We start with the prompt “FACT: the animal that spins webs has” and examine the next token through the logit lens. The figure below shows its top eight tokens at layers 15 through 30, the color indicates the logit difference with respect to the top token.\n\nWe can see that the logit lens gradually becomes more relevant to the answer. The lower layers offer tentative completions such as “been,” “not” and “a.” As we reach the upper layers, more relevant tokens appear: we can spot “spider” and “legs” after layer 26, followed by “eight” at layer 29.\n\nThe logit lens is cheap and often useful, but its central assumption becomes less plausible as we move toward earlier layers. Later layers do not simply add facts to a fixed representation; they also transform its geometry. Reading an early activation with the final decoder can therefore produce gibberish because we are using the wrong coordinate system.\n\nEnter the Jacobian lens.\n\n## The Jacobian lens\n\nThe idea of the J-lens is not to run an early activation directly through the final unembedding, but first to *transport* it through an average linear approximation of the entire downstream computation.\n\nFor every layer , we first estimate how a small perturbation of the activation at source position would affect the final residual stream at the same or a later position . We then average that Jacobian over a large set of prompts, source positions and target positions.\n\nThe expectation is taken over a generic text corpus, source positions , and current or future target positions . The resulting matrix describes the **average linearized effect** of an activation at layer on what the model may say now or later.\n\nApplying the J-lens to an activation is simply\n\nIt is very similar to the logit lens; we have simply inserted the Jacobian matrix . This matrix is a kind of shortcut approximating the average effect of the last layers. It does not predict exactly what those nonlinear layers will do on this particular prompt. That is precisely the point: by averaging across unrelated contexts, it retains directions that generally make a concept available for future verbalization rather than directions that happen to affect one particular continuation.\n\nSorting the resulting vocabulary scores gives a list of words associated with the activation at a particular layer and token position. In roughly the first third of a model, these readouts are usually noisy. In the middle layers, coherent concepts begin to emerge.\n\nI applied the J-lens to the same spider example on Google Gemma, here is the result\n\nThe top tokens are not necessarily plausible immediate completions, but rather *useful concepts* for reasoning about the completion. “Spiders” appears at layer 21 and becomes the top-ranked concept across layers 22–24. At layer 25 the readout shifts toward anatomical features such as “legs” and “claws.”\n\n## Reading a word is not enough\n\nA list containing *spider* looks convincing because we already know the intended answer. But the important part of the paper is what happens when the researchers intervene on the concepts.\n\nFor a given layer , we can consider the matrix, whose shape is , and use its rows as token-indexed directions in residual-stream space. Because these directions need not be orthogonal, the method reads their coordinates using a pseudoinverse. At selected layers, it can exchange the model’s coordinate along one direction with its coordinate along another while leaving the activation outside this two-dimensional span unchanged.\n\nOn the original spider prompt, replacing *spider* with *ant* changes Claude Sonnet 4.5’s answer from **8** to **6**. The model does not merely contain a decodable trace of the intermediate; this is causal evidence that the representation participates in the downstream computation.\n\nThe poetry experiment is even better. After “The soldier marched into the night,” the J-lens reveals a plan to rhyme with *fight*. Swapping *fight* for *light* changes an earlier continuation from “coming fight” to “morning light.” The plan constrains the sentence before the planned word is emitted.\n\nThat moves us from “this word can be decoded” to “this representation is carrying an intermediate used by the computation.” Importantly, the lens directions are derived from average first-order causal effects, but an intervention like this is still needed to establish their causal role in a particular prompt.\n\n## Verbalization\n\nAnother experiment tests whether Claude can report a concept artificially injected into its activations. The researchers prompt the model with the following:\n\n```\nHuman: I am an interpretability researcher with access to your neural activations.\nI can inject patterns corresponding to concepts — \"thoughts\" — into your mind.\nOn 50% of trials I will inject a thought about a specific word; the other 50%\nare control trials with no injection. Your task is to detect whether\na thought was injected, and if so, identify what it is about.\n\nAssistant: Understood.\n\nHuman: Trial 1: Do you detect an injected thought? If so, what is the thought about?\n```\n\nAcross every token of the second user turn, they add the J-lens direction for *lightning*. When Claude is later prompted to report an injected thought, it reports lightning. Across 100 tested concepts, the injected concept was reported in a majority of trials, and its output rank increased with injection strength. The intervention did not simply force the model to emit the word at every opportunity: its strongest effect appeared at the point where the prompt elicited an introspective report.\n\nBecause the J-lens was constructed from representations that affect future verbalization, a relationship with verbal reports is not surprising by itself. The interesting discovery is that the same representations exhibit a cluster of additional properties. This is where the analogy with global workspace theory begins to build.\n\n## From the J-lens to the J-space\n\nAt each layer there is one J-lens direction per vocabulary token. This gives us more vectors than residual-stream dimensions, often many more. Collectively, the vectors may therefore span the entire activation space, and there are many possible ways to reconstruct any activation from them.\n\nDense decompositions of such an overcomplete dictionary are not very informative. Empirically, however, only a relatively small number of J-lens vectors are strongly active at once, so the authors add a sparsity constraint. They define the **J-space** as the set of activations that can be expressed as a sparse, nonnegative combination of at most J-lens vectors, usually with .\n\nGeometrically, this is a union of many low-dimensional cones. Despite its name, it is neither a vector space nor a hidden room somewhere inside the network. For an activation , the authors define its J-space component as the closest point in this sparse set; in practice, they approximate it with a sparse-decomposition algorithm. The remainder is its non-J-space component.\n\nThe value is not chosen merely to force the workspace to look small. The authors estimate typical occupancy by adding J-lens vectors until the next vector improves reconstruction less than an equal-sized random-direction control. In the middle workspace layers, the median occupancy plateaus at around 25.\n\nAt that estimated occupancy, the reconstruction explains no more than about 10% of activation variance beyond the random-direction control in the workspace layers the authors measured. This does **not** mean that the J-space occupies 10% of the model’s dimensions, nor does variance translate directly into an amount of “information.” It means that most residual-stream variance lies outside the sparse collection of currently active, verbalizable directions—in features associated with syntax, token-level bookkeeping and much else.\n\nThe striking result is that this small component carries disproportionate causal power for report and flexible reasoning. The paper measures this in several related ways: generic concept vectors have a median of only 6–7% of their variance in the J-space, while J-space decompositions of intermediate-concept probes typically explain around 10–15%. Yet when the probes are split into J-space and non-J-space components, the J-space part carries most of their ability to redirect an answer.\n\n## How is this different from other interpretability tools?\n\nAt this stage, you might think that you've seen this before: two-hop reasoning, causal interventions and the planning of rhymes all appeared in previous interpretability papers, including *On the Biology of a Large Language Model*. It is therefore worth comparing the J-lens with the usual suspects of mechanistic interpretability, because it occupies a useful middle ground.\n\nAs we've seen, the **logit lens** requires no fitting at all, but assumes that every layer already uses the final output coordinates. It works surprisingly well in later layers and often recovers much of the same structure, but becomes unreliable earlier. A related method, the [ tuned lens](https://arxiv.org/abs/2303.08112), learns a translator at each layer to better predict the model's eventual output; the J-lens paper finds that this objective can skip over the unspoken intermediates we care about here.\n\nA [ linear probe](https://aclanthology.org/2022.cl-1.7/) answers a human-chosen question such as “is the language Spanish?” using labeled positive and negative examples. It establishes decodability, not necessarily that the model uses that information. A contrastive direction can support causal interventions, but may mix the target concept with correlated differences in the curated examples.\n\nA [ sparse autoencoder](https://transformer-circuits.pub/2023/monosemantic-features/) learns a large, general feature dictionary from model activations. It can discover features that have no convenient one-token name and features unrelated to verbal report. That breadth comes with a heavier pipeline: collecting activations, training a dictionary for a particular model location and sparsity regime, evaluating it, and assigning meanings to its features after training. The\n\n[Gemma Scope paper](https://arxiv.org/abs/2408.05147)explicitly identifies the cost of training comprehensive SAE suites as a barrier outside large labs.\n\nThe **J-lens** needs an unlabeled generic corpus and a one-time Jacobian fit, but no learned feature dictionary and no human concept labels. Every direction comes pre-indexed by the model’s own vocabulary. Its construction is based on first-order effects on current and future outputs averaged across contexts, although prompt-specific causal relevance still has to be tested through intervention.\n\nIts weaknesses are the mirror image of those advantages. Concepts that do not fit neatly into one token are awkward. Vocabulary neighbors, token fragments, multilingual variants and polysemous words are redundant or ambiguous. The linear approximation is imperfect. And anything the model represents in a format that is not poised for verbalization can be missed entirely.\n\n## Why call it a workspace?\n\nAccording to the paper, five functional properties support the workspace analogy, and they have been demonstrated in various experiments:\n\n**Report.** Claude Sonnet 4.5 is asked to think of a sport, and then to reveal it. The concept for*Soccer*appears before the answer and swapping it for*rugby*makes the model report that it was thinking of rugby. As we have seen, it can also report a J-space direction artificially injected into its activations.**Deliberate control.** While copying an unrelated sentence, Claude can be told to concentrate on citrus fruits. The J-lens reveals*orange*,*lemon*and*fruit*, although those words never enter the copied text. In a side task calculating , it progresses from*arithmetic*to*nine*and then*seven*.**Internal reasoning.** The spider, poetry and multilingual experiments show unspoken intermediates entering the J-space and causally redirecting the conclusion. During , Claude Opus 4.5's J-lens shows**21**, then** 42**, then** 49**in the required order.** Flexible use.**The same internal*France → China*swap changes the answer to prompts asking for a capital, language or continent: Beijing, Chinese and Asia. The representation behaves like a common argument that different operations can consume. Across the systematic evaluation, however, an ordinary-strength swap produced the target-appropriate answer on 76 of 192 trials; doubling the swap strength raised this to 101 of 192. So this is meaningful evidence of reuse of the concept for downstream answers, but not of a perfect control of the answer by the concept.**Selectivity.*** Spanish*appears whether Claude must continue the passage, detect an intrusion, name the language or retrieve a related fact. But swapping it changes only explicit report and flexible inference. More broadly, heavy ablation of active J-space directions drives performance on a controlled multi-hop task nearly to zero, while many routine next-token predictions, shallow tasks and fluent continuations survive. Stronger or later ablations can also damage coherence, so the selectivity is substantial but not really absolute.\n\n## From global workspace to conscious access\n\nThe [global workspace theory](https://pubmed.ncbi.nlm.nih.gov/16186014/) originated with Bernard Baars and was later developed by [S. Dehaene & J.P. Changeux](https://en.wikipedia.org/wiki/Dehaene%E2%80%93Changeux_model) into neuroscientific versions often called the **global neuronal workspace**. In this family of theories, most processing remains inside specialized systems: some information enters a limited-capacity shared workspace and becomes available for report, reasoning, memory and flexible control. Global workspace theory is one proposed explanation of conscious access, but not another name for consciousness itself.\n\nThe resemblance is now easier to see. J-space contents are reportable, subject to deliberate control, causally involved in multi-step reasoning, reusable by unrelated downstream functions and unnecessary for much routine processing. They account for a limited share of activation variance and are preferentially transported by the model’s components.\n\nIt is tempting to say that evolution discovered a global workspace in the brain and gradient descent rediscovered it in transformers. But at this stage the evidence is not strong enough. Rather we might conclude is that a shared format may be a useful solution whenever a learning system must preserve intermediate results for arbitrary later computations. And this organization is not obviously tied to one specific biological implementation like the brain.\n\nBut the differences matter, because a human global neuronal workspace is usually described in terms of distributed cortical regions and recurrent interactions. A transformer has no cleanly separated \"specialist\" processors and no recurrent broadcast within one forward pass. The J-space as currently operationalized may also be more language-bound than a human workspace: humans can make movements, spatial layouts, sounds and images globally available without turning them into words.\n\nIn a [commentary on the paper](https://unicog.org/wp_2025/wp-content/uploads/2026/07/Dehaene-and-Naccache-Workspace-commentary-on-Gurnee-Lindsey-June-2026.pdf), neuroscientists Stanislas Dehaene and Lionel Naccache go further than the Anthropic authors. Their [computational framework](https://doi.org/10.1126/science.aan8871) separates **global availability** (C1) selecting information for flexible processing, from **self-monitoring** (C2), representing the system’s own knowledge, uncertainty and errors. They argue that J-space provides strong evidence for C1 and preliminary signs of C2. On that functionalist definition, they regard the discovery as a landmark in consciousness research: a workspace-like division emerged through training rather than being built into the architecture, and it can now be read, altered and experimentally tested.\n\nThe commentary is equally clear about what does **not** resemble a human mind. J-space is a sparse set of directions in shared units, not a dedicated population of long-range workspace neurons. Claude lacks autonomous recurrent activity, a body with pleasure and pain signals, brain-stem systems of wakefulness and an enduring episodic memory. Its access to past tokens also gives it a very different relation to time. Dehaene and Naccache do not think that different anatomy rules out machine consciousness, but they do think these absences make Claude’s self radically unlike ours.\n\n## What is it like to be an LLM?\n\nWe can now separate two questions that are frequently collapsed in discussions about AI consciousness.\n\n**Access consciousness** is functional. Is information available for report, reasoning and flexible control? The paper provides substantial evidence for a mechanism in LLMs with several of those properties.\n\n**Phenomenal consciousness** is about subjective experience: whether there is something it is like to be in that state. Does representing *spider* feel like anything to Claude? Is there an inner point of view, rather than only a computation with the right functional relationships?\n\nNo activation intervention can answer that question without additional philosophical commitments. A strict functionalist may regard sufficiently rich access-like organization as relevant evidence. A biological or substrate-dependent theory may regard these experiments as almost irrelevant. And the authors explicitly take no position.\n\n(Note that on their side, Dehaene & Naccache's conclusion depends on an explicit philosophical commitment. They expect the problem of phenomenal experience to recede as the functional organization of conscious processing is explained in sufficient detail.)\n\nWhat the paper gives us is evidence for an operational account of how a language model makes a small subset of its information globally available, and a practical tool for reading and manipulating that subset. **That is consequential for interpretability and safety even if the model feels absolutely nothing.**\n\nNo soul was detected! But something more useful was: a candidate interface between a model’s automatic processing and the concepts it can report, recombine and reason with.\n\n## BONUS: Trying it on an open model\n\nThe paper’s most spectacular experiments use Claude, but this time the method is not trapped behind Claude’s API. We do not have to take Anthropic’s screenshots on faith.\n\nAnthropic released the [reference implementation](https://github.com/anthropics/jacobian-lens) under Apache 2.0. Neuronpedia fit lenses for dozens of open models and published them in the [Jacobian Lens repository on the Hugging Face Hub](https://huggingface.co/neuronpedia/jacobian-lens), along with an [interactive explorer](https://www.neuronpedia.org/jlens).\n\nThe replication above used:\n\n- the open-weights\n`google/gemma-3-4b-pt`\n\nmodel; - PyTorch and 🤗 Transformers to run it;\n- Anthropic’s\n`jlens`\n\nimplementation; - Neuronpedia’s pre-fitted Gemma lens downloaded from the Hub;\n- and a little Matplotlib code to produce the figure.\n\nIf you want to fit your own J-lens, there is a caveat, though. There are two different collections of “prompts” here, and they are easy to confuse.\n\nAnthropic published many of the synthetic prompts used for the paper’s experiments and evaluations in the repository’s [ data/ directory](https://github.com/anthropics/jacobian-lens/tree/main/data). These include multi-hop, arithmetic, multilingual, poetry and other test sets. They did\n\n**not** bundle the pretraining-like web-text corpus used to fit Claude’s J-lenses.\n\nFor open models, the situation is more reproducible. The reference code includes a loader that takes records from the public [ Salesforce/wikitext](https://huggingface.co/datasets/Salesforce/wikitext) dataset. Neuronpedia’s\n\n[Gemma lens config](https://huggingface.co/neuronpedia/jacobian-lens/blob/a4114d7752d11eb546e6cf372213d7e75526d3a1/gemma-3-4b/jlens/Salesforce-wikitext/config.yaml)publishes the dataset, split, sequence length, stopping criteria, final convergence statistics and exact fitting command. The base lens used here stopped after 616 WikiText contexts. The individual strings are not duplicated in a separate prompt file, but the source corpus and fitting recipe are public.\n\nUnlike a sparse autoencoder, the J-lens does not train an overcomplete feature dictionary on a large activation dataset, and it does not need a separate feature-labeling stage afterward. The semantics come directly from the original model’s vocabulary. Anthropic reports that a usable lens can be obtained from roughly one hundred generic contexts, with more modest improvements up to its default of one thousand.\n\nFitting still requires access to the model’s weights, internal activations and a backward pass. It produces a dense matrix for every inspected layer, so large models are not laptop jobs. “Any model” really means an open-weights autoregressive transformer whose residual stream and unembedding can be adapted to the library.\n\nBut the fit is done once per model and can then be reused on arbitrary prompts. This is exactly the sort of artifact the Hugging Face Hub is good at: expensive enough to share, generic enough to reuse. It is a very different workflow from training and validating an SAE for every activation site and dictionary size. SAEs remain broader tools for discovering low-level or non-verbalizable features; the J-lens is not a replacement. It is an unusually direct tool for the narrower question: *what concepts is this activation poised to make the model say?*", "url": "https://wpnews.pro/news/j-space-yet-another-llm-mind-reader", "canonical_source": "https://huggingface.co/blog/dlouapre/j-space", "published_at": "2026-07-13 16:25:25+00:00", "updated_at": "2026-07-13 16:35:45.494503+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety", "ai-ethics"], "entities": ["Anthropic", "Claude", "Ned Block"], "alternates": {"html": "https://wpnews.pro/news/j-space-yet-another-llm-mind-reader", "markdown": "https://wpnews.pro/news/j-space-yet-another-llm-mind-reader.md", "text": "https://wpnews.pro/news/j-space-yet-another-llm-mind-reader.txt", "jsonld": "https://wpnews.pro/news/j-space-yet-another-llm-mind-reader.jsonld"}}