{"slug": "neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits", "title": "Neuron Explorer for Qwen3.5-4B: See every neuron at work, make targeted edits", "summary": "A steering-first Neuron Explorer for Qwen3.5-4B was released, pushing every MLP neuron in a layer up and down across a few prompts, keeping every rollout, and laying the results over a 3D view of the model. The explorer loads Qwen3.5-4B locally on MPS, CUDA or CPU for live steered chat, lens, connections and inference, and its Claude tab runs the claude CLI with the explorer as an MCP server (explorer/agent_mcp.py) that can read neurons, cones and lenses, run the live model, and move the map and highlights. The sweep pipeline runs on a CUDA 13 GPU pod with the steering fork of vLLM (RhizoNymph/vllm), using main_rel doses as fractions of the median residual norm leaving the layer: {who, snake, orange} × {−0.7, +0.5, +0.8} and {free} × {+3}, plus one unsteered baseline per prompt.", "body_md": "Steering-first neuron explorer for Qwen3.5-4B. Every MLP neuron in a layer is pushed up and down on a few prompts, every rollout is kept, and the explorer lays the results out over a 3D view of the model.\n\n```\nuv sync\nuv run python explorer/server.py        # http://localhost:8791\n```\n\nReads sweep runs from `runs/` and per-layer stats from `stats/`, both produced by the sweep below; the caches in\n`explorer/cache/` ship with the repo. Loads Qwen3.5-4B locally (MPS, CUDA or CPU) for live\nsteered chat, lens, connections and live inference over your own text.\n\n- **map** : the model's layers in perspective, attention and MLP per layer, one point per neuron. Hover shows a neuron's\nsteer lens and logit lens; click opens it; click a layer's label to view its sheet face-on.`writes` places\nneurons with similar down-projection columns next to each other.\n- **list** : units ranked by word-cluster mass against random-direction controls;`lens` searches the steer and logit\nlenses for a token,`fires` runs your text through the model and filters neurons by activation.\n- **neuron** : steered rollouts per prompt and dose, a three-column multi-turn steered chat (−, 0, +), lens and\nconnections, and a name.\n- **embeddings** : UMAP of a layer's rollouts, coloured by neuron, prompt, dose or kind.\n- **Claude** (fourth neuron-panel tab, or the right-edge handle): a chat running the`claude` CLI (installed and\nlogged in) with the explorer as an MCP server (`explorer/agent_mcp.py` ). It reads neurons, cones and lenses, runs the\nlive model (steering, activations, next-token logprobs, ablations), moves the map, list and highlights, and has file\ntools and a shell in your home folder. Its prompt is`explorer/agent_prompt.md` plus`explorer/agent_methodology.md` , and`explorer/agent_prompt.local.md` when present (untracked, for your own additions);`EXPLORER_CLAUDE_MODEL` picks the model. The server answers only pages and tools on this machine.\n\nRuns on a CUDA 13 GPU pod with the steering fork of vLLM ([RhizoNymph/vllm](https://github.com/RhizoNymph/vllm)).\n\n```\nbash sweep/setup_pod.sh                                   # fork, deps, model; writes /workspace/env.sh\npython sweep/neuron_stats.py --layers 22 --tokens 2000000 --out stats/q35-4b\npython sweep/steer_sweep.py --stats stats/q35-4b_L22.npz --design main_rel --dose-unit rel \\\n    --neurons all --random-controls 256 --run q35-4b-L22\nsweep/run_depth.sh \"4 9 13 17\" 384 64                     # random-subset sweep across several layers\n```\n\n`main_rel` doses are fractions of the median residual norm leaving the layer: {who, snake, orange} × {−0.7, +0.5, +0.8}\nand {free} × {+3}, plus one unsteered baseline per prompt.\n\n| script | output | \n|---|---|\n| `sweep/neuron_stats.py` | per-neuron activation extremes by sign regime, down columns, residual norms | \n| `sweep/steer_sweep.py` | `runs/<run>.jsonl` : text, token ids, logprobs, top-k per step; optional activation captures | \n| `sweep/rank.py` | `runs/<run>.units.jsonl` : distinctive words, cluster mass, percentile against controls | \n| `sweep/steer_lens.py` | `stats/steer_lens_L{l}.npz` : next-token Δlogprob and KL for ±0.8 pushes on every neuron | \n| `sweep/depth_report.py` | `runs/depth_report.json` : per-layer share of neurons above controls, coherence, top units | \n| `sweep/shift.py` | word-shift score against controls, all words counted | \n| `sweep/merge_runs.py` | joins sharded runs of one layer | \n| `sweep/check_equivalence.py` | HF vs vLLM steering and capture agreement | \n| `sweep/bench.py` ,`sweep/peek.py` | throughput conditions; terminal views of a run | \n| `explorer/build_embeddings.py` | `runs/<run>.umap.json` | \n| `explorer/build_layout.py` | `explorer/cache/layout_writes.npz` | \n| `explorer/build_connections.py` | `explorer/cache/connections.npz` : each neuron's 24 strongest direct-path links up and down | \n| `sweep/capture_prefill.py` | `captures/<run>.mlp_in.npy` : MLP input at the last prompt token at the run's capture layers, steered | \n\n`research_log.md` holds the findings and reasoning; `NOTES.md` the source notes.", "url": "https://wpnews.pro/news/neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits", "canonical_source": "https://github.com/lyramakesmusic/neuron-explorer", "published_at": "2026-10-26 19:46:54+00:00", "updated_at": "2026-10-26 20:11:45.304947+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-research", "ai-agents", "developer-tools"], "entities": ["Qwen3.5-4B", "Neuron Explorer", "Claude", "vLLM", "RhizoNymph/vllm", "MCP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits", "markdown": "https://wpnews.pro/news/neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits.md", "text": "https://wpnews.pro/news/neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits.txt", "jsonld": "https://wpnews.pro/news/neuron-explorer-for-qwen3-5-4b-see-every-neuron-at-work-make-targeted-edits.jsonld"}}