{"slug": "tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers", "title": "TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers", "summary": "Developer Sudharsanselvaraj built TokenPrint, an open-source 3D visual debugger that lets users inspect the internal computation of transformer and LLM inference layer by layer. The tool treats inference as a journey, exposing operations such as Q/K/V projections, attention scores, and residual connections, and labels each element with provenance categories — REAL, DERIVED, CONCEPTUAL, or SIMULATION — to distinguish actual model data from illustrative content.", "body_md": "Large language models are usually presented as a stack of equations, architecture diagrams, and source code.\n\nThat explains what the model is.\n\nBut it doesn't always explain **what is actually happening to a token as it moves through the model**.\n\nSo I built **TokenPrint**.\n\n**Try it:** [https://tokenprint.in/](https://tokenprint.in/)\n\n**GitHub:** [https://github.com/Sudharsanselvaraj/Token-Print](https://github.com/Sudharsanselvaraj/Token-Print)\n\nTokenPrint is an open-source interactive visualization and debugging environment for transformer and LLM inference.\n\nThe goal is simple:\n\n**Make the internal computation of a transformer something you can actually see and inspect.**\n\nInstead of looking at:\n\n`x → attention → MLP → x`\n\nyou can explore the individual operations that produce that result.\n\nYou can move through the model layer by layer and inspect things such as:\n\nOne of the ideas behind TokenPrint is to treat inference as a journey.\n\nA token starts as a discrete token ID.\n\nIt becomes an embedding.\n\nThat representation enters the first transformer layer.\n\nThen it passes through normalization, attention, projections, nonlinear transformations and residual connections.\n\nThe representation continues evolving through the remaining layers until the model produces its final logits.\n\nTokenPrint makes that process explorable in 3D.\n\nYou can select operations, move between layers, follow the computational path, and inspect the associated tensors and metadata.\n\nFor example, an attention block can be understood as a sequence of transformations:\n\n**Q projection**\n\n`Q = XW_Q`\n\n**K projection**\n\n`K = XW_K`\n\n**V projection**\n\n`V = XW_V`\n\nThen positional information is applied to the relevant representations before attention scores are calculated.\n\nThe attention mechanism can then be expressed as:\n\n`A = softmax(QKᵀ / √d_k + M)`\n\nand the context representation becomes:\n\n`C = AV`\n\nFinally, the concatenated head outputs are projected back into the model's hidden dimension.\n\nFor a model using Grouped-Query Attention, TokenPrint can also expose the relationship between query heads and shared key/value groups instead of treating the attention block as a single opaque operation.\n\nA transformer is already a computational graph.\n\nThe problem is that a traditional diagram becomes difficult to read once you start showing many layers, branches and intermediate tensors.\n\nTokenPrint uses 3D space to make those relationships spatial.\n\nYou can move around the computation, focus on a layer, inspect an operation, and follow the data path.\n\nThe intention isn't to make the model look futuristic.\n\nThe intention is to make the structure easier to understand.\n\nSelecting a component opens an inspection view containing information such as:\n\n**What it does**\n\nA plain-language explanation of the operation.\n\n**Equation**\n\nThe mathematical operation represented by the component.\n\n**Dimensions**\n\nInput and output dimensions.\n\n**Parameters**\n\nThe number of learned parameters involved.\n\n**Model data**\n\nThe actual tensor path associated with the component.\n\n**Provenance**\n\nWhether the displayed information is real model data, derived information, conceptual structure, or simulation.\n\nFor example, selecting a projection can show a tensor such as:\n\n`model.layers.11.self_attn.k_proj.weight`\n\nalong with its shape, dtype, parameter count, layer and runtime information.\n\nA major design goal of TokenPrint is to distinguish between what the model actually provides and what is only being illustrated.\n\nThe visualizer uses explicit provenance categories:\n\n**REAL** — directly obtained from the model/runtime.\n\n**DERIVED** — calculated from real model information.\n\n**CONCEPTUAL** — an educational representation of a model concept.\n\n**SIMULATION** — intentionally simulated behavior.\n\nThis distinction matters because an attractive visualization is not useful if it quietly invents model internals.\n\nTokenPrint also provides a tensor inspection workflow.\n\nInstead of treating tensors as invisible implementation details, you can inspect their metadata directly.\n\nFor example:\n\n`model.layers.3.self_attn.v_proj.weight`\n\ncan expose:\n\nShape: `128 × 896`\n\nDtype: `float32`\n\nParameters: approximately `114.7K`\n\nLayer: `3`\n\nRuntime: `hf_local`\n\nThat creates a connection between the visualization and the actual model implementation.\n\nTokenPrint is also intended to be an educational tool.\n\nA component should answer more than:\n\n\"What is this box?\"\n\nIt should explain:\n\n**What does this operation compute?**\n\n**Why does the transformer need it?**\n\n**What is its input?**\n\n**What is its output?**\n\n**How does it connect to the next operation?**\n\n**Which parameters and tensors are involved?**\n\nThis makes it possible to move from a high-level explanation to the mathematical and implementation-level details without leaving the visualization.\n\nThe long-term goal is not only visualization.\n\nTokenPrint is being designed as an environment for inspecting and experimenting with transformer computation.\n\nThat includes areas such as:\n\nThe idea is to make the internal behavior of an LLM inspectable rather than treating inference as a black box.\n\nTokenPrint is open source and built for people who want to learn, experiment, research, and contribute.\n\nThere are opportunities across:\n\nGitHub:\n\n[https://github.com/Sudharsanselvaraj/Token-Print](https://github.com/Sudharsanselvaraj/Token-Print)\n\nI'm working toward making TokenPrint much more than a visualization of one local model.\n\nSome of the directions include broader Hugging Face model support, better model capability detection, richer execution traces, remote inference, experiment workflows, and shareable transformer traces that can be explored without reproducing the original runtime.\n\nThe larger idea is:\n\n**What if opening an LLM did not just show you the model's architecture, but let you actually watch computation happen inside it?**\n\nThat's what I'm trying to build with TokenPrint.\n\n**Try TokenPrint → [https://tokenprint.in/](https://tokenprint.in/)**\n\n**Star the project or contribute → [https://github.com/Sudharsanselvaraj/Token-Print](https://github.com/Sudharsanselvaraj/Token-Print)**\n\nExplore what happens inside a transformer: follow tokens through embeddings, attention, Q/K/V projections, RoPE, MLPs, residual streams, and logits with an interactive 3D visual debugger.", "url": "https://wpnews.pro/news/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers", "canonical_source": "https://dev.to/sudharsan_nef/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers-2jca", "published_at": "2026-09-12 15:01:13+00:00", "updated_at": "2026-09-12 15:14:31.796802+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-research"], "entities": ["TokenPrint", "Sudharsanselvaraj", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers", "markdown": "https://wpnews.pro/news/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers.md", "text": "https://wpnews.pro/news/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers.txt", "jsonld": "https://wpnews.pro/news/tokenprint-an-open-source-3d-visual-debugger-for-llms-and-transformers.jsonld"}}