{"slug": "interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models", "title": "Interactive Jacobian-Lens visualizer and live steerer for GGUF models", "summary": "A new open-source tool, jlens-gguf, implements Anthropic's Jacobian Lens for GGUF models, enabling interactive visualization, live steering, and concept manipulation of language model internals via a web UI and llama.cpp-based server.", "body_md": "A GGUF-native implementation of the **Jacobian Lens** from Anthropic's\n[ Verbalizable Representations Form a Global Workspace in Language\nModels](https://transformer-circuits.pub/2026/workspace/index.html), plus an\n\n**interactive visualizer & manipulator**(in the spirit of Neuronpedia's J-Lens pages) that works against GGUF models served by llama.cpp — with\n\n**live steering, concept swapping (patching), ablation, and steered generation**.\n\n```\n                                  browser (web UI)\n                                        │  JSON\n                        ┌───────────────▼────────────────┐\n                        │  bridge  (python, numpy)       │  lens math:\n                        │  jlens_gguf.server             │  J_l transport, unembed,\n                        │   • model GGUF → readout wts   │  top-k, ranks, J-lens vectors,\n                        │   • lens GGUF  → J_l (+ bias)  │  steering / swap / ablate,\n                        └───────┬────────────────────────┘  sparse decomposition\n                                │  HTTP (binary activations, generic residual edits)\n                        ┌───────▼────────────────────────┐\n                        │  jlens-server  (C++, llama.cpp)│  the same GGUF file your\n                        │   • captures l_out-<il> per    │  llama-server mmaps — weights\n                        │     layer/position             │  are shared page cache\n                        │   • applies add / low-rank /   │\n                        │     set edits mid-graph        │\n                        │   • generation w/ edits active │\n                        └────────────────────────────────┘\n```\n\nNew here? Start with— a friendly, task-oriented walkthrough. This README is the full reference.[QUICKSTART.md]\n\nThe reference PyTorch implementation reads the residual stream with hooks and\ncomputes with torch tensors. Here, **the lens lives in a GGUF file**, readout\nweights (final norm + unembedding) are **read directly from the model GGUF**\n(dequantized with gguf-py, no torch), and residual-stream access + live\ninterventions go through a small **llama.cpp-based activation server** that\nhooks the computation graph via ggml's scheduler eval callback.\n\n```\n# clone with the llama.cpp submodule\ngit clone --recursive <your-repo-url> jlens-gguf   # or: git clone … && git submodule update --init\n\n# 0) prerequisites: gcc, cmake, python3  (llama.cpp comes as a submodule)\n\n# 1) build the native activation server (builds llama.cpp too if needed)\nnative/build.sh\n\n# 2) python env\npython3 -m venv .venv && .venv/bin/pip install -e .   # or: pip install numpy requests gguf\n\n# 3) fit a lens for your model (ridge regression over WikiText, ~minutes on CPU)\n.venv/bin/python -m jlens_gguf fit \\\n    --model models/qwen2.5-1.5b-instruct-q8_0.gguf \\\n    --corpus wikitext:100 \\\n    -o lenses/qwen1.5b-lens.gguf\n\n# 4) launch the visualizer (autostarts the native server)\n.venv/bin/python -m jlens_gguf serve \\\n    --model models/qwen2.5-1.5b-instruct-q8_0.gguf \\\n    --lens lenses/qwen1.5b-lens.gguf\n# → open http://127.0.0.1:8090\n```\n\nOr all-in-one: `scripts/jlens-up MODEL.gguf [LENS.gguf]`\n\n.\n\nIf you already have a **llama-server** running, `quickstart`\n\nreads which GGUF\nit serves (from its `/props`\n\n), spins up the introspection sidecar on the same\nfile, auto-loads a matching lens if one exists, and opens the browser:\n\n```\npython -m jlens_gguf quickstart --llama-server http://127.0.0.1:8080\n# or just a file:\npython -m jlens_gguf quickstart models/qwen2.5-1.5b-instruct-q8_0.gguf\n# with no args it tries http://127.0.0.1:8080\npython -m jlens_gguf quickstart\n```\n\n(The sidecar mmaps the same GGUF, so this adds only its KV cache, not another copy of the weights.)\n\nPrompt: `Fact: The capital of Japan is Tokyo.\\nFact: The currency used in the country shaped like a boot is`\n\n— the model answers *\" the Japanese yen\"*: it\nbinds \"currency\" to the salient *Japan* instead of the boot-shaped country.\nThe lens shows exactly this: ` yen`\n\ncollapses to rank 0 in the last layers\nwhile ` euro`\n\n/` Euro`\n\nhover at rank 60–500 in the workspace band — the\nrunner-up hypothesis the paper describes. Add two **ablate** interventions\n(` yen`\n\nand ` Japanese`\n\n, layers 14–26) and generate:\n\nbaseline:`the Japanese yen.`\n\n→ablated:`the Euro.`\n\na concept-level correction of a faulty two-hop, performed live on a quantized GGUF through llama.cpp.\n\nWithout `--lens`\n\nthe visualizer runs with the **identity lens** (= the classic\nlogit lens) — everything works, readouts are just less faithful at early\nlayers.\n\nFitting and visualization work on **any GGUF llama.cpp can load** — dense or\nMoE (Qwen3-MoE, Mixtral, DeepSeek, OLMoE, …). The lens only ever touches the\n`d_model`\n\n-wide residual stream (`l_out-<il>`\n\n), which is identical whether the\nblock's FFN is dense or a sparse expert mixture, so **MoE routing and expert\ncount don't affect the lens at all**. Quantized weights are fine: readout\nweights are dequantized to fp32 and the numpy readout matches llama.cpp's own\nlogits to corr > 0.9999 even at Q4 (verified on a 64-expert OLMoE and on Q8\nQwen).\n\n**Scaling to large models.** The regression fit's memory is\n`n_layers × 2 × d_model² × itemsize`\n\n(two Gram matrices per fitted layer) and\nthe lens file is `n_layers × d_model² × 2`\n\nbytes — both `O(n_layers·d_model²)`\n\nand **independent of expert count**. `fit`\n\nprints this footprint up front. For\na ~200–400B MoE (e.g. Qwen3.5-397B-A17B): the *model itself* dominates memory,\nso run it on hardware sized to load it; the lens then adds ~25–50 GiB of fit\nRAM (use `--gram-dtype float32`\n\nto halve it) and a multi-GiB lens file. If that\ndoesn't fit at once, fit a **band** of layers with `--layers a,b,c`\n\nover\nseveral passes and combine them with `JacobianLensGGUF.merge`\n\n. The main\n*interactive* cost is the readout grid (`positions × d_model × vocab`\n\nper\nlayer) — use a layer **stride** or shorter prompts on very large models. Per-\ntoken forward cost is set by the *active* parameters (A17B ≪ 397B), so capture\npasses stay tractable, and GPU offload (`-ngl`\n\n) speeds them up.\n\nThe paper's estimator `J_l = E[∂h_final/∂h_l]`\n\nneeds backprop, which llama.cpp\ndoes not have. Two options:\n\n-\n— per-layer ridge regression`fit`\n\n(GGUF-native, above)`h_final ≈ A_l h_l + b`\n\nover a corpus: the same-position,*correlational*surrogate of the Jacobian transport (a tuned-lens-style translator). Works on any quantized GGUF, forward passes only. -\n— fit the true causal lens on the original HF checkpoint with`convert-pt`\n\n(exact)[Anthropic's reference code](https://github.com/anthropics/jacobian-lens), then convert the`.pt`\n\nto lens-GGUF (works*without*torch installed):\n\n```\n.venv/bin/python -m jlens_gguf convert-pt lens.pt lens.gguf\n```\n\nThe converted lens drops into the same visualizer.\n\n`jlens-server`\n\nis also a **drop-in llama-server replacement**: point any\nOpenAI-compatible app (Open WebUI, SillyTavern, your own code, the `openai`\n\nSDK) at it, and a **server-held live intervention set** is applied to *every*\ncompletion it serves — so you can steer, ablate, or swap concepts in the\ntokens the app actually generates, and watch it in the visualizer beside it.\n\n```\n   your chat app ──OpenAI──▶  jlens-server /v1/chat/completions\n                                     ▲ applies the live intervention set\n   J-Lens UI  ──push/clear──────────┘   (steer / swap / ablate)\n              ──\"load last chat\"──▶  visualize the exact conversation\n```\n\nEndpoints on `jlens-server`\n\n(default `:8091`\n\n):\n\n| endpoint | purpose |\n|---|---|\n`POST /v1/chat/completions` |\nOpenAI chat, streaming + non-streaming |\n`POST /v1/completions` |\nOpenAI text completion |\n`GET /v1/models` |\nmodel id |\n`GET/POST/DELETE /jlens/interventions` |\nread / set / clear the live set |\n`GET /jlens/last_completion` |\nthe exact tokens of the last completion |\n\nInterventions apply live during both prompt processing and token generation\n(they edit the residual stream mid-graph, so they affect the KV cache and the\nsampled tokens). In the UI's **Live backend** panel:\n\n**push interventions**— install the enabled chips as the backend's live set (position ranges become all-positions, since an app's conversation doesn't line up with visualizer prompts).**clear**— remove them; the backend returns to normal.** load last chat**— pull the app's most recent completion into the heatmap (prompt + generated tokens marked) to see the readouts behind what it just said.\n\nExample — steer a downstream app with nothing but the stock `openai`\n\nclient:\n\n``` python\nimport openai\nclient = openai.OpenAI(base_url=\"http://127.0.0.1:8091/v1\", api_key=\"x\")\n\n# operator (elsewhere): POST /jlens/interventions to ablate the \" Euro\" direction\nclient.chat.completions.create(model=\"m\", messages=[\n    {\"role\": \"user\", \"content\": \"The currency of the country shaped like a boot is the\"}])\n# → \"…is the Indonesian rupiah\" instead of \"…the Euro\": the concept is gone,\n#   live, and the output stays grammatical.\n```\n\n**Prefix caching / determinism:** the backend reuses the KV prefix across\nturns (a chat app's growing history decodes only the new tokens). Like\nllama-server, greedy output is *not* bit-invariant to prefix reuse — llama.cpp\nsums cached vs in-batch KV blocks in a different order, so a completion may\ndiffer from a fully-fresh recompute by a token. The **visualizer's** forward\npath always recomputes fresh, so its readouts are exact and reproducible;\nchanging the live set invalidates the cache automatically.\n\n**What it can't do:** attach to an *already-running* stock `llama-server`\n\nprocess — activation access needs the callback registered at context creation,\nwhich is why the sidecar exists. Run your app against `jlens-server`\n\ninstead;\nit mmaps the same GGUF, so weights are shared.\n\nThere are two integration levels, depending on whether you want to *watch* the\nj-space or *manipulate* what your app generates:\n\n**(a) Inspect only — leave your app untouched.** Keep your app on its current\nllama-server. Run `jlens-gguf quickstart --llama-server <its-url>`\n\n: it reads\nthe model from that server and opens the visualizer on the same GGUF. You get\nthe full layer×position readout, pins, rank charts, and decomposition for any\nprompt you paste in — no change to your app. (It can't see the app's *live*\nactivations, since those live in the other process; it recomputes them on the\nsame weights.)\n\n**(b) Inspect and steer your app — point your app at jlens-server.**\nBecause\n\n`jlens-server`\n\nspeaks the OpenAI API and accepts llama-server's launch\nflags, it's a drop-in backend:\n\n```\n# however you launch llama-server today, e.g.:\n#   llama-server -m model.gguf -c 8192 -ngl 99 --host 0.0.0.0 --port 8080\n# just swap the binary — same flags work:\nnative/jlens-server -m model.gguf -c 8192 -ngl 99 --host 0.0.0.0 --port 8080\n```\n\nNow point your app's base URL at `http://<host>:8080/v1`\n\n(for talk-llama.cpp,\nthat's its OpenAI/server endpoint; for an agent framework like **ATHENA**, set\nthe model/base-URL it uses for llama.cpp to this). Open the visualizer\n(`jlens-gguf serve --llama-server http://<host>:8080`\n\nor point `--native-url`\n\nat it), build steer/swap/ablate interventions, and hit **push interventions** —\nthey apply to every completion your app requests from that moment on, and\n**load last chat** shows you the readouts behind what it just said. Remove them\nwith **clear** and the backend behaves like a normal llama-server again.\n\nSo: *ATHENA (or any llama-server client) → jlens-server → your GGUF*, with the\nJ-Lens UI riding alongside. If you tell me exactly how ATHENA points at\nllama.cpp (a base URL, a spawned\n\n`llama-server`\n\n, or an in-process\n`libllama`\n\n), I can tailor a drop-in recipe or a launch wrapper.`jlens-server`\n\n**does not fork or patch llama.cpp or ggml** — it links against\ntheir *public* API only (`llama.h`\n\n, `ggml.h`\n\n, `ggml-backend.h`\n\n, `libllama`\n\n),\nusing the same `ggml_backend_sched`\n\neval-callback hook that `llama-imatrix`\n\nand the debug tooling use. The HTTP and JSON dependencies are vendored under\n`native/vendor/`\n\n, so the build never reaches into llama.cpp's internal file\nlayout either. Updating is therefore just:\n\n```\ncd llama.cpp && git checkout <newer-tag-or-commit> && cd ..   # or: git submodule update --remote\nnative/build.sh          # rebuilds libllama + jlens-server\n```\n\nllama.cpp is pinned as a **git submodule** (currently `ggml-org/llama.cpp`\n\nat a\nknown-good commit), so a fresh clone gets a matching pair, and bumping it is one\ncommand. The one internal convention `jlens-server`\n\nrelies on is the residual\ntensor name `l_out-<il>`\n\n(stable across all mainstream decoder architectures);\na startup self-check reports `l_out_ok: false`\n\nin `/props`\n\nif a future/unusual\narchitecture ever stops exposing it, so a mismatch fails loudly rather than\nsilently.\n\n**Heatmap**— lens top-1 token at every (position, layer); scrollable, virtualized, hover for the full top-k. Click a cell to select it and pin its token.`⇧`\n\n+hover scrubs; arrow keys move the selection.**By-Layer / By-Pos panels**— full top-k readout down the stack at the selected position, and across positions at the selected layer.** Pins**— pinned tokens get log-rank trajectories (rank vs layer, rank vs position) and a viridis rank heatmap over the whole grid.**Cell readout**— top-40 tokens with probabilities at the selected cell; one click steers/swaps with the cell's top token.** Decompose**— the paper's sparse J-space decomposition (greedy matching pursuit onto J-lens vectors): which token-directions make up this activation, with coefficients and explained variance.**Interventions**(the manipulator):** steer**—`h ← h + α‖h‖·v̂_t`\n\nat chosen layers/positions. Positive α summons a concept, negative suppresses it.**swap**— the paper's concept patch: read lens coordinates`c = V⁺h`\n\nof two tokens and exchange them,`h ← h + V(σ(c)−c)`\n\n. The component orthogonal to both directions is untouched.**ablate**— project a token's J-lens direction out of the stream.- Interventions apply\n**live** during prompt processing*and*generation; cells whose top-1 changed vs. the un-intervened baseline get an orange marker, with the baseline token shown in the tooltip.\n\n**Generate**— continue the prompt with interventions active, side-by-side with the un-steered baseline; or \"visualize continuation\" to extend the heatmap into generated tokens.**lens checkbox**— toggle between the fitted lens and the raw logit lens.\n\nURL params: `?prompt=...&autorun=1&gen=24`\n\n.\n\n**Native server** (`native/jlens_server.cpp`\n\n, ~700 lines, links only\n`libllama`\n\n): registers a `ggml_backend_sched`\n\neval callback. llama.cpp names\neach block's residual output `l_out-<il>`\n\n; when such a node finishes, the\ncallback (a) applies any interventions to the requested (layer, position)\nranges by editing the tensor in place — downstream nodes, the KV entries of\nlater layers, and the model's own sampled tokens all see the edit — and (b)\ncopies the (post-edit) activations out. Three edit primitives cover all lens\noperations: `add`\n\n(steering vector), `lowrank`\n\n(`h += A(Bh)`\n\n— swap is rank-2,\nablation rank-1), and `set`\n\n. One detail worth knowing: llama.cpp gathers only\nlogit-requested rows before the last block, so the server requests logits at\nall positions whenever the final layer is captured.\n\n**Bridge** (`jlens_gguf/server.py`\n\n): reads `output.weight`\n\n/ `output_norm.*`\n\nstraight from the model GGUF (any quantization; dequantized once to fp32),\nloads `J_l`\n\nfrom the lens GGUF, and does all lens math in numpy:\n\n```\nlens_logits(h, l) = softcap( W_U · norm( J_l h + b_l ) )\n```\n\nJ-lens vectors are `v_t = J_lᵀ(γ ⊙ W_U[t])`\n\n(the norm's diagonal scale folded\nin). The UI's steer/swap/ablate specs are translated into native `add`\n\n/\n`lowrank`\n\nedits per fitted layer, with steering magnitudes scaled by the\nlayer's typical residual norm (stored in the lens file at fit time).\n\n**Lens GGUF format**: tensors `jlens.J.{layer}`\n\n`[d,d]`\n\n(fp16) and optional\n`jlens.b.{layer}`\n\n; metadata `jlens.d_model`\n\n, `jlens.source_layers`\n\n,\n`jlens.target_layer`\n\n, `jlens.fit_method`\n\n(`jacobian`\n\n| `regression`\n\n|\n`identity`\n\n), `jlens.n_prompts`\n\n, `jlens.h_rms`\n\n. `python -m jlens_gguf inspect lens.gguf`\n\nprints it.\n\n`jlens-server`\n\naccepts the common **llama-server** launch flags so you can swap\nthe binary in place: `-m/--model`\n\n, `-c/--ctx-size`\n\n, `-b/--batch-size`\n\nand\n`-ub/--ubatch-size`\n\n(both map to the prompt chunk), `-t/--threads`\n\n,\n`-ngl/--n-gpu-layers`\n\n, `-mg/--main-gpu`\n\n, `-fa/--flash-attn [on|off|auto]`\n\n,\n`--no-mmap`\n\n, `--mlock`\n\n, `--host`\n\n, `--port`\n\n. Single-sequence-only flags it\ndoesn't need (`--parallel`\n\n, `--cont-batching`\n\n, `--api-key`\n\n, `--jinja`\n\n, …) are\naccepted and ignored with a notice, so an existing launch command won't error.\n`jlens-server --help`\n\nlists them.\n\n`jlens-server`\n\nis built from your llama.cpp checkout and **mmaps the same\nGGUF** your `llama-server`\n\nserves, so model weights are shared page cache —\nrunning both costs one copy of the weights plus each server's KV cache. Run\nyour chat frontend against llama-server as usual and point the visualizer at\nthe same file to inspect/steer the same model live.\n\n```\nGET  /props /vocab /health /v1/models\nPOST /tokenize /detokenize /apply_template\nPOST /jlens/forward   {tokens, capture_layers?, dtype?, interventions?,\n                       n_predict?, sampling?, logits_positions?}\n      → \"JLNS\" binary: header JSON + raw f16/f32 activation blocks\nPOST /v1/chat/completions /v1/completions   (OpenAI-compatible; live set applied)\nGET/POST/DELETE /jlens/interventions        (the live intervention set)\nGET  /jlens/last_completion                 (exact tokens of the last completion)\n```\n\nInterventions: `{layer, pos_start, pos_end (-1 = ∞), mode: add|set|lowrank, data: b64 f32, k?}`\n\n. See `native/jlens_server.cpp`\n\nheader comment for the\nfull schema, and `jlens_gguf/client.py`\n\nfor the Python client.\n\n```\n.venv/bin/python -m pytest tests/    # 60 tests (incl. backend mode)\n```\n\nCovers: lens GGUF roundtrip, pure-python `.pt`\n\nconversion, numpy unembed vs\nllama.cpp logits (≤1e-4), capture determinism, all three edit primitives\nagainst closed-form expectations, regression-fit quality vs the logit lens,\nevery bridge endpoint including steering/swap/ablate rank effects, **backend\nmode** (OpenAI chat/completions, streaming, stop sequences, live-intervention\neffects, KV prefix reuse, forward/completion interplay), and a\nheadless-Chromium drive of the full UI. `tests/test_moe.py`\n\nadds a\nMixture-of-Experts check (capture + readout + intervention + steered\ngeneration), auto-skipped unless a MoE GGUF is present\n(`JLENS_MOE_MODEL=/path/to/moe.gguf`\n\n).\n\n- The regression lens is a\n*correlational*approximation; for the paper's causal claims, fit with the reference code and`convert-pt`\n\n. - Grid computation is\n`T × d × vocab`\n\nper layer in numpy — a few seconds per run for a 1.5B model with a 152k vocab on CPU. Use layer`stride`\n\nor shorter prompts if it drags. - Architectures whose graphs don't name\n`l_out`\n\ntensors are rejected at startup (`l_out_ok: false`\n\nin`/props`\n\n) — all mainstream llama.cpp decoder archs have them. `apply_template`\n\nuses llama.cpp's built-in template matcher (chatml fallback), not a full Jinja engine.- Bind addresses default to 127.0.0.1; neither server has auth. Keep them local.\n\n```\nnative/           jlens_server.cpp, build.sh, poc_cb.cpp\n  vendor/         vendored cpp-httplib + nlohmann/json (build is self-contained)\njlens_gguf/       python package: lens, model_reader, readout, client,\n                  fitting, pt_convert, server (bridge), cli, web/ (UI + d3)\ntests/            pytest suite + CDP browser driver\nscripts/jlens-up  one-command launcher\nllama.cpp/        git submodule (ggml-org/llama.cpp, public API only)\nlenses/           fitted lenses (gguf; gitignored — fit or download locally)\n```\n\nCode: Apache-2.0 (see `LICENSE`\n\n/`NOTICE`\n\n). The web UI's heatmap/panel/rank-chart\ndesign is adapted from Anthropic's jacobian-lens reference visualization\n(Apache-2.0). Vendored `cpp-httplib`\n\nand `nlohmann/json`\n\nare MIT; `d3`\n\nis ISC.", "url": "https://wpnews.pro/news/interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models", "canonical_source": "https://github.com/igorbarshteyn/jlens-gguf", "published_at": "2026-07-13 07:42:54+00:00", "updated_at": "2026-07-13 08:05:27.965489+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-tools", "ai-research"], "entities": ["Anthropic", "Neuronpedia", "llama.cpp", "GGUF", "Qwen2.5-1.5B-Instruct"], "alternates": {"html": "https://wpnews.pro/news/interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models", "markdown": "https://wpnews.pro/news/interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models.md", "text": "https://wpnews.pro/news/interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models.txt", "jsonld": "https://wpnews.pro/news/interactive-jacobian-lens-visualizer-and-live-steerer-for-gguf-models.jsonld"}}