{"slug": "show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations", "title": "Show HN: Dolittle, explore and steer Qwen3's internal activations", "summary": "Plainsight Systems LLC released Dolittle, an open-source C++20 engine that runs Qwen3 8B alongside marco-bazzani's natural language autoencoder, capturing the output of block 24 (zero-based) so users can read text descriptions of the model's internal activations, edit those descriptions, and compare the model's continuation with and without the change. The tool runs generation, interpretation and steering on CUDA and Metal through one browser interface, requires Python 3.11+ and Git, and downloads about 45 GB of model files on first launch; it is available on Linux/NVIDIA (tested on an RTX 3090 Ti with 24 GB VRAM) and macOS/Apple Silicon (macOS 26+, tested on an M3 Max with 128 GB unified memory), with a native Windows/NVIDIA build planned and AMD undecided. A historical 4B/8B reference comparison run in PyTorch used 15.4 GiB peak allocation with the models in separate BF16 phases, and the project is described as R&D under Plainsight Systems LLC.", "body_md": "Dolittle puts Qwen3’s generated thinking beside natural language explanations of its internal activations. Enter a prompt, select a token and read what the NLA describes. Edit that explanation, then compare the model’s continuation with and without the change.\n\nThe C++20 engine runs **Qwen3 8B with marco-bazzani’s natural language\nautoencoder**, capturing the output of block 24 (zero-based). Generation,\ninterpretation and steering run on CUDA and Metal through the same browser\ninterface. Each experiment keeps the original response, interpretation samples\nand steering comparisons available to explore.\n\nUse Python 3.11+ and Git, then run the same command on either supported host:\n\n```\ngit clone https://github.com/plainsight-systems/dolittle.git\ncd dolittle\npython3 scripts/run.py\n```\n\n| Platform | Status and launch path | Tested hardware | \n|---|---|---|\n| Linux/NVIDIA | Available. Builds and runs in Docker with NVIDIA Container Toolkit. | RTX 3090 Ti, 24 GB VRAM | \n| macOS/Apple Silicon | Available. Builds and runs natively with Metal. Requires macOS 26+, Xcode 26 with Metal toolchain, CMake 3.24+ and OpenSSL 3. | M3 Max, 128 GB unified memory | \n| Windows/NVIDIA | Native build planned. | — | \n| AMD | Undecided. | — | \n\nThe launcher builds from source, downloads about 45 GB of model files and checks\ntheir hashes. Later launches reuse those files and the saved verification result.\nUse `--verify-models` to check every file again and repair the cache.\n\nOpen the full **Open Dolittle** link printed by the launcher and keep that\nterminal running. The public project website explains the experiment; the local\nservice runs it. A browser on another machine can connect to that service.\nSee [application setup](https://github.com/plainsight-systems/dolittle/blob/main/app/README.md) for prerequisites, cache options and remote\naccess, then follow the [rhyming walkthrough](https://dolittle.plainsight-systems.com/writing/running-an-investigation/).\n\nThe engine uses BF16 tensors and FP32 reductions. It keeps model weights on the GPU between requests when there is room for the next model, its cache and working buffers. Each generation gets a fresh KV cache. CUDA builds include native Ampere code and PTX that a compatible NVIDIA driver can compile for newer cards.\n\nDevelopment checks:\n\n```\npython3 scripts/check.py host\n# On Linux: build CUDA sources in the pinned Docker environment.\npython3 scripts/check.py cuda-build\n```\n\nSee [build and verification commands](https://github.com/plainsight-systems/dolittle/blob/main/tests/README.md) and\n[VS Code setup](https://github.com/plainsight-systems/dolittle/blob/main/tests/README.md#vs-code-and-compiler-settings). The historical\n[4B/8B reference comparison](https://github.com/plainsight-systems/dolittle/blob/main/docs/research/2026-10-16-nla-comparison/README.md)\nused PyTorch to compare the model pairs before building this engine. The 8B\nrun used 15.4 GiB peak allocation with the models running in separate BF16 phases.\n\nThe project site can be previewed independently:\n\n```\npython3 scripts/site.py serve\n```\n\nOpen [http://127.0.0.1:1315/](http://127.0.0.1:1315/). Python 3.11+ and Hugo 0.154.2 are required; see\n[site setup](https://github.com/plainsight-systems/dolittle/blob/main/web/README.md). The production URL is [https://dolittle.plainsight-systems.com/](https://dolittle.plainsight-systems.com/). GitHub Pages\nbuilds and deploys the site on pushes to `main`.\n\nDoctor Dolittle learned to communicate with animals. This project takes its name from that idea: making an unfamiliar intelligence understandable.\n\nA natural language autoencoder translates an internal activation vector into text, then reconstructs a vector from that text. Its descriptions are interpretations that can be wrong. Dolittle distinguishes those interpretations from the reasoning text the target model generates itself.\n\nThe purpose is to understand the machinery by implementing it and making it easy to interact with. Dolittle is R&D under Plainsight Systems LLC.\n\n| Path | Purpose | \n|---|---|\n| `src/` | C++20, CUDA and Metal inference engine | \n| `app/` | Browser interface served with the engine | \n| `web/` | Public project website | \n| `tests/` | Numerical, GPU, and application verification | \n| `scripts/` | Source-build, setup, launch, and check tooling | \n| `docs/architecture/` | Logical design, change axes, and file mapping | \n| `docs/process/` | Project memory, work queue, and local workflow | \n| `docs/research/` | Source investigations and measurements | \n| `docs/writeups/` | Explanations, explorations, and milestone articles | \n| `docs/assets/` | Project artwork and its provenance | \n\nDownloaded weights live in ignored `models/`; build output and local\nexperiments also stay out of Git.\n\nStart with [project memory](https://github.com/plainsight-systems/dolittle/blob/main/docs/process/MEMORY.md), the\n[work queue](https://github.com/plainsight-systems/dolittle/blob/main/docs/process/QUEUE.md), and the\n[local workflow](https://github.com/plainsight-systems/dolittle/blob/main/docs/process/workflow.md). Agent entry points are\n[`AGENTS.md`](https://github.com/plainsight-systems/dolittle/blob/main/AGENTS.md) and [` CLAUDE.md`](https://github.com/plainsight-systems/dolittle/blob/main/CLAUDE.md).\n\nThe [architecture overview](https://github.com/plainsight-systems/dolittle/blob/main/docs/architecture/README.md) explains the computation\nand how the CUDA and Metal backends share it. [Verification](https://github.com/plainsight-systems/dolittle/blob/main/tests/README.md)\ncovers numerical comparisons, tests and build checks.\n\n- [Anthropic's natural language autoencoder article](https://www.anthropic.com/research/natural-language-autoencoders)\n- [Natural Language Autoencoders paper](https://transformer-circuits.pub/2026/nla/index.html)\n- [Original NLA implementation and checkpoints](https://github.com/kitft/natural_language_autoencoders)\n- [Selected Qwen3 8B NLA pair](https://huggingface.co/marco-bazzani/Qwen3-8B-nla)\n\nCode and configuration use Apache-2.0; documentation and original artwork use\nCC BY 4.0. See [LICENSE](https://github.com/plainsight-systems/dolittle/blob/main/LICENSE) for scope and [NOTICE](https://github.com/plainsight-systems/dolittle/blob/main/NOTICE) for attribution.\nThird-party weights and components retain their own licenses.", "url": "https://wpnews.pro/news/show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations", "canonical_source": "https://github.com/plainsight-systems/dolittle", "published_at": "2026-10-22 14:49:47+00:00", "updated_at": "2026-10-22 15:12:07.126285+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools", "ai-safety"], "entities": ["Dolittle", "Plainsight Systems LLC", "Qwen3 8B", "marco-bazzani", "NVIDIA", "Apple", "CUDA", "Metal"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations", "markdown": "https://wpnews.pro/news/show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations.md", "text": "https://wpnews.pro/news/show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations.txt", "jsonld": "https://wpnews.pro/news/show-hn-dolittle-explore-and-steer-qwen3-s-internal-activations.jsonld"}}