{"slug": "the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian", "title": "The Complete Guide to Reading a Model’s Hidden Layers with Anthropic’s Jacobian Lens", "summary": "Anthropic's July 2026 paper 'Verbalizable Representations Form a Global Workspace in Language Models' introduces the Jacobian lens (J-lens), a tool that corrects the basis mismatch causing the older logit lens to fail in early layers, and demonstrates that a low-dimensional slice of hidden layers, J-space, acts as a global workspace. In tests, the model silently recognized a blackmail scenario in this space before writing a reply, and suppressing that space made it more likely to attempt the blackmail, providing causal evidence of its role. The companion library jlens, installable from GitHub, enables fitting lenses and visualizing results on a free Colab T4.", "body_md": "When a language model answers a question, most of the computation happens in the middle of the network(*the hidden layers)*, long before you see the answer.\n\nAnd knowing exactly what happens in those hidden layers is one of the harder problems in mechanistic interpretability, largely because the standard techniques like the logit lens fail due to a basis mismatch.\n\nSo, in July 2026, Anthropic published *Verbalizable Representations Form a Global Workspace in Language Models*, arguing that a small, low-dimensional slice of those hidden layers— what they call **J-space** — behaves like a global workspace. This global workspace means the concepts or “thoughts” a model is leaning toward saying, even if it never actually says them out loud.\n\nFor example: In the paper, researchers gave the model an email-based blackmail scenario as a test. Before the model wrote a single word of its reply, this hidden space already contained words like “blackmail” and “fake” — meaning the model had silently recognized the setup and was already weighing that response, even though its final written reply never mentioned either idea.\n\nWhen the researchers then suppressed that hidden space and reran the same scenario, the model’s behaviour actually changed: it became more likely to attempt the blackmail it had previously only “considered” internally. That’s the causal evidence — not just correlation — that this hidden space is doing real work in the model’s decision.\n\nThat’s a genuinely new kind of interpretability result, and it hinges on a new tool: the **Jacobian lens (J-lens)**, which corrects for the basis mismatch that causes the older logit lens to produce garbage in early layers.\n\nThis article stays narrower than the paper. We’ll load a small open model, apply both the logit lens and the Jacobian lens to identical activations, watch where the correct answer to a simple factual question first becomes legible, and fit a lens of our own — all runnable on a free Colab T4.\n\nIn this article, we’ll look into the hidden layers of a Qwen model using the **Jacobian lens**. We’ll compare it directly against the older logit lens on identical activations, plot where the correct answer emerges, and fit a lens from scratch.\n\n*Note: The entire setup runs on a free Colab T4, so you can follow along without paying for compute.*\n\nTo understand the Jacobian Lens, we must first examine the limitations of the baseline **Logit Lens**.\n\nThe logit lens is the original approach to this problem. It takes a hidden state from some intermediate layer and pushes it straight through the model’s output matrix, as if the layer you’re probing were the last one.\n\nThe logit lens relies on a strong assumption: that downstream layers won’t significantly change the representation. Early in the network, this assumption fails. Because in a 30-layer model, layer 5 is still 25 layers away from the output space and hasn’t been rotated into the expected vocabulary basis. Forcing this unrefined state through the unembedding matrix produces garbled output such as punctuation marks, unrelated foreign characters, or strings of underscores.\n\nThe Jacobian lens replaces that identity assumption with a learned correction. For each layer, it fits a matrix that approximates what the remaining layers actually do, then applies that matrix before unembedding.\n\nBecause the correction is fit by averaging real gradients across many prompts, it captures what a given activation is *generally* disposed to push the model toward, rather than what happened in one specific context.\n\njlens is the companion library released with the paper. It handles fitting lenses, applying them to prompts, and visualising the results. It also ships the interactive slice-stack viewer used in the paper's figures.\n\n*Note: It isn’t published to PyPI, so it installs from GitHub at the time of this writing.*\n\nLet h_l be the hidden state at layer l (a d-dimensional vector), and W_U be the unembedding matrix (mapping d hidden dimensions to vocabulary size V).\n\nThe **logit lens** decodes h_l by applying W_U directly:\n\n```\nP_logit = softmax( W_U · LayerNorm(h_l) )\n```\n\nThe **Jacobian lens** fits a projection matrix J_l (dimension d × d), defined as the expected gradient of the final hidden state h_L with respect to the probed layer state h_l:\n\n```\nJ_l ≈ E[ ∂h_L / ∂h_l ]\n```\n\nThe hidden state is transported through J_l before unembedding:\n\n```\nP_jacobian = softmax( W_U · LayerNorm(J_l · h_l) )\n```\n\nThat single matrix multiplication is the entire architectural difference between the two methods.\n\nThe diagram below shows the three paths out of the same hidden state: the model’s real forward pass, the logit lens shortcut, and the Jacobian lens shortcut.\n\nBefore starting, make sure your environment has the following:\n\nYou can find the [Colab notebook here](https://colab.research.google.com/drive/1Tcq_pKdaZMdCleaxacUuOJeD0uwrjT37?usp=sharing)\n\nI used Qwen/Qwen3.5-4B in this tutorial, which is about 8GB in bfloat16 and ungated. Larger models cause two problems on a T4. meta-llama/Meta-Llama-3-8B is gated, so loading it without an accepted license and a token raises OSError: You are trying to access a gated repo. It is also roughly 16GB in bf16, which is the full capacity of a T4, so it raises an out-of-memory error while moving weights to the GPU even after authentication is sorted out.\n\nIf you want to use a larger or gated model, you’ll need a bigger GPU, 4-bit quantization, or both.\n\nInstall the required packages. jlens installs from its [GitHub repository](https://github.com/anthropics/jacobian-lens) since it isn't on PyPI.\n\n```\n!pip install -q torch transformers pandas accelerate matplotlib!pip install -q git+https://github.com/anthropics/jacobian-lens#egg=jlens\n```\n\nNext, log in to [Hugging Face](https://huggingface.co). This is required for gated models and avoids download rate limits on ungated ones.\n\n``` python\nfrom huggingface_hub import loginlogin()\n```\n\nLoad the base model and wrap it for jlens. The library works through its own wrapper rather than the raw Hugging Face model object.\n\n``` python\nimport torchimport pandas as pdfrom transformers import AutoModelForCausalLM, AutoTokenizerimport jlens\nmodel_id = \"Qwen/Qwen3.5-4B\"tokenizer = AutoTokenizer.from_pretrained(model_id)hf_model = AutoModelForCausalLM.from_pretrained(    model_id,    dtype=torch.bfloat16,).cuda()model = jlens.from_hf(hf_model, tokenizer)\n```\n\nNote that the keyword is dtype, not torch_dtype. jlens pins a version of transformers that renamed this argument, and the older name will throw.\n\nA fitted lens is one matrix per layer, stored as a .pt file. Pre-fitted lenses for several models are hosted on the Hub, so there's no need to fit your own to get started.\n\n```\nlens = jlens.JacobianLens.from_pretrained(    \"neuronpedia/jacobian-lens\",    filename=\"qwen3.5-4b/jlens/Salesforce-wikitext/Qwen3.5-4B_jacobian_lens_n1000.pt\",    revision=\"qwen-n1000\",)\n```\n\nTwo things to keep in mind here.\n\nThe function below reads the final token position at every layer the lens supports, running the same activations through both lenses.\n\n```\nprompt = \"The capital of the country where the Eiffel Tower is located is\"\npython\ndef probe_trajectory(model, lens, prompt, tokenizer):    # Restrict probing to layers the loaded lens actually covers    layers = sorted(set(range(1, model.n_layers)) & set(lens.source_layers))    logits_jac, _, _ = lens.apply(        model, prompt, layers=layers, positions=[-1], use_jacobian=True    )    logits_std, _, _ = lens.apply(        model, prompt, layers=layers, positions=[-1], use_jacobian=False    )    results = []    for l in layers:        top_std = tokenizer.decode([logits_std[l][0].argmax().item()])        top_jac = tokenizer.decode([logits_jac[l][0].argmax().item()])        results.append({\"Layer\": l, \"Logit Lens\": top_std, \"Jacobian Lens\": top_jac})    return pd.DataFrame(results)df = probe_trajectory(model, lens, prompt, tokenizer)print(df.iloc[1:30])\n```\n\nThepositions=[-1] reads the last token, which is the point just before the model commits to an answer. The use_jacobian flag is the whole comparison: same forward pass, same weights, same activations, with only the transport step changing.\n\nAs you can see, the logit lens never recovers. Across all 29 layers shown, it produces no French word, no place name, nothing geography-adjacent, but the Jacobian lens gets it on the 26th layer. This is the failure the Jacobian lens was built to address, and it appears exactly as described.\n\nThe Jacobian lens is also mostly noise. Its column is full of ... and ____, so this is not a method that makes every intermediate layer legible.\n\nAt layers 26 and 30, though, it returns “Paris.” Those are the only two cells in the table, in either column, where the correct answer surfaces before the model’s final output.\n\nIt’s worth being precise about what this supports. A single top-1 hit is not a picture of what the model was thinking at that instant, and most layers here remain unreadable. What the table does support is the comparison: on identical activations, one method surfaced the answer twice, and the other surfaced it zero times.\n\nEvaluating only Top-1 argmax tokens masks continuous probability shifts in intermediate layers. Tracking the absolute vocabulary rank of a target token on a logarithmic scale provides a clear metric of emerging confidence across network depth.\n\n``` python\nimport matplotlib.pyplot as plt\npython\ndef token_rank_trajectory(model, lens, prompt, target_token, tokenizer):    layers = sorted(set(range(1, model.n_layers)) & set(lens.source_layers))    target_id = tokenizer.encode(target_token, add_special_tokens=False)[0]    logits_jac, _, _ = lens.apply(        model, prompt, layers=layers, positions=[-1], use_jacobian=True    )    logits_std, _, _ = lens.apply(        model, prompt, layers=layers, positions=[-1], use_jacobian=False    )    ranks_jac, ranks_std = [], []    for l in layers:        ranks_jac.append(            (logits_jac[l][0].argsort(descending=True) == target_id).nonzero().item()        )        ranks_std.append(            (logits_std[l][0].argsort(descending=True) == target_id).nonzero().item()        )    return layers, ranks_jac, ranks_stdlayers, ranks_jac, ranks_std = token_rank_trajectory(    model, lens, prompt, target_token=\" Paris\", tokenizer=tokenizer)plt.figure(figsize=(9, 5))plt.plot(layers, ranks_std, marker=\"o\", label=\"Logit lens\", color=\"tab:gray\")plt.plot(layers, ranks_jac, marker=\"o\", label=\"Jacobian lens\", color=\"tab:blue\")plt.yscale(\"log\")plt.gca().invert_yaxis()plt.xlabel(\"Layer\")plt.ylabel('Rank of \" Paris\" (log scale)')plt.title(\"Rank of the correct answer across layers\")plt.legend()plt.grid(alpha=0.3)plt.show()\n```\n\nNote the leading space in \" Paris\". This tokenizer, like most, encodes a space-prefixed word as a different token from the bare word. If a rank curve looks flat and wrong, check this first.\n\nNext, we can also visualise representations across the entire prompt sequence and layer stack, because it jlens integrates interactive visualizers and dictionary gloss mappings used in the Jacobian Lens paper.\n\n```\n!mkdir -p assets!wget -q https://raw.githubusercontent.com/anthropics/jacobian-lens/main/assets/qwen_gloss.json.gz -P assets/\npython\nimport urllib.request, gzip, json, os# Download Qwen token gloss lookup tableURL = \"https://raw.githubusercontent.com/anthropics/jacobian-lens/main/assets/qwen_gloss.json.gz\"os.makedirs(\"assets\", exist_ok=True)urllib.request.urlretrieve(URL, \"assets/qwen_gloss.json.gz\")print(os.path.getsize(\"assets/qwen_gloss.json.gz\"))   # expect 655852gloss = {int(k): v for k, v in json.load(gzip.open(\"assets/qwen_gloss.json.gz\")).items()}print(len(gloss), \"glosses\")                          # expect 91695\nslice_data = compute_slice(model, lens, prompt, layer_stride=2, mask_display=True)page, _, _ = build_page(    slice_data,    prompt,    title=\"Eiffel Tower probe\",    description=\"Multi-hop factual recall, probed at the final token position.\",    alt_token=gloss,)notebook_iframe(page)\n```\n\nSetting mask_display=True filters raw output tokens into human-readable subwords, removing noise such as standalone punctuation or formatting tokens. You can click on any cell pin to track its rank across every layer at once.\n\nA public copy of this viewer with pre-loaded examples is available at[transformer-circuits.pub/2026/workspace/public/slice-stack/](http://transformer-circuits.pub/2026/workspace/public/slice-stack/). It is a useful reference for what a strong readout looks like compared to the mixed one above. For longer prompts, switch to mode=\"fetch\", which writes rank data to sidecar files instead of inlining it into the page.\n\nTwo constraints are worth carrying forward, particularly given how mixed the Step 5 output looked.\n\n**It’s a linear approximation of a nonlinear system.** J_l is a single matrix standing in for everything downstream — attention routing included — and it can't represent the sharp, discontinuous decisions attention makes. The expectation in E[∂h_final/∂h_l] is taken across many contexts, which means it offers no guarantee for any one particular context. This is plausibly part of why most layers in the table were unreadable.\n\n**A lens doesn’t transfer.** Not across base models, and not across fine-tunes of the same base. Fine-tuning changes internal representations, so J_l has to be recomputed. The fitting procedure in Step 8 is identical regardless of the model, but it costs compute every time.\n\n**It only captures single-token concepts cleanly, and mid-network depths specifically.** Multi-token ideas and very early- or very-late-layer content are harder for this method to surface — the useful readouts concentrate in the middle third to two-thirds of the network’s depth.\n\nThe Jacobian lens doesn’t give you a clean window into a transformer’s intermediate reasoning — on the prompt tested here, most layers stayed unreadable under both methods.\n\nBut what it does provide is a readout that can surface the correct answer at points where the logit lens structurally cannot, on identical activations.\n\nThat’s also the mechanism behind the paper’s larger claim: that a specific, low-dimensional subspace of a model’s activations holds the concepts it’s currently poised to verbalise, separate from the much larger volume of automatic processing happening in parallel — and that this subspace can be read, and in some cases edited, before the model ever writes a word.\n\nFor interpretability and safety work where the alternative is decoding noise, that difference is the point.\n\n[The Complete Guide to Reading a Model’s Hidden Layers with Anthropic’s Jacobian Lens](https://pub.towardsai.net/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropic-s-jacobian-lens-c564c9ac4fcf) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian", "canonical_source": "https://pub.towardsai.net/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropic-s-jacobian-lens-c564c9ac4fcf?source=rss----98111c9905da---4", "published_at": "2026-08-22 17:01:01+00:00", "updated_at": "2026-08-22 17:13:20.901741+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research", "ai-tools"], "entities": ["Anthropic", "Jacobian lens", "J-lens", "jlens", "Qwen", "Colab T4"], "alternates": {"html": "https://wpnews.pro/news/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian", "markdown": "https://wpnews.pro/news/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian.md", "text": "https://wpnews.pro/news/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian.txt", "jsonld": "https://wpnews.pro/news/the-complete-guide-to-reading-a-models-hidden-layers-with-anthropics-jacobian.jsonld"}}