TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers

Developer Sudharsanselvaraj built TokenPrint, an open-source 3D visual debugger that lets users inspect the internal computation of transformer and LLM inference layer by layer. The tool treats inference as a journey, exposing operations such as Q/K/V projections, attention scores, and residual connections, and labels each element with provenance categories — REAL, DERIVED, CONCEPTUAL, or SIMULATION — to distinguish actual model data from illustrative content.

Large language models are usually presented as a stack of equations, architecture diagrams, and source code. That explains what the model is. But it doesn't always explain what is actually happening to a token as it moves through the model . So I built TokenPrint . Try it: https://tokenprint.in/ https://tokenprint.in/ GitHub: https://github.com/Sudharsanselvaraj/Token-Print https://github.com/Sudharsanselvaraj/Token-Print TokenPrint is an open-source interactive visualization and debugging environment for transformer and LLM inference. The goal is simple: Make the internal computation of a transformer something you can actually see and inspect. Instead of looking at: x → attention → MLP → x you can explore the individual operations that produce that result. You can move through the model layer by layer and inspect things such as: One of the ideas behind TokenPrint is to treat inference as a journey. A token starts as a discrete token ID. It becomes an embedding. That representation enters the first transformer layer. Then it passes through normalization, attention, projections, nonlinear transformations and residual connections. The representation continues evolving through the remaining layers until the model produces its final logits. TokenPrint makes that process explorable in 3D. You can select operations, move between layers, follow the computational path, and inspect the associated tensors and metadata. For example, an attention block can be understood as a sequence of transformations: Q projection Q = XW Q K projection K = XW K V projection V = XW V Then positional information is applied to the relevant representations before attention scores are calculated. The attention mechanism can then be expressed as: A = softmax QKᵀ / √d k + M and the context representation becomes: C = AV Finally, the concatenated head outputs are projected back into the model's hidden dimension. For a model using Grouped-Query Attention, TokenPrint can also expose the relationship between query heads and shared key/value groups instead of treating the attention block as a single opaque operation. A transformer is already a computational graph. The problem is that a traditional diagram becomes difficult to read once you start showing many layers, branches and intermediate tensors. TokenPrint uses 3D space to make those relationships spatial. You can move around the computation, focus on a layer, inspect an operation, and follow the data path. The intention isn't to make the model look futuristic. The intention is to make the structure easier to understand. Selecting a component opens an inspection view containing information such as: What it does A plain-language explanation of the operation. Equation The mathematical operation represented by the component. Dimensions Input and output dimensions. Parameters The number of learned parameters involved. Model data The actual tensor path associated with the component. Provenance Whether the displayed information is real model data, derived information, conceptual structure, or simulation. For example, selecting a projection can show a tensor such as: model.layers.11.self attn.k proj.weight along with its shape, dtype, parameter count, layer and runtime information. A major design goal of TokenPrint is to distinguish between what the model actually provides and what is only being illustrated. The visualizer uses explicit provenance categories: REAL — directly obtained from the model/runtime. DERIVED — calculated from real model information. CONCEPTUAL — an educational representation of a model concept. SIMULATION — intentionally simulated behavior. This distinction matters because an attractive visualization is not useful if it quietly invents model internals. TokenPrint also provides a tensor inspection workflow. Instead of treating tensors as invisible implementation details, you can inspect their metadata directly. For example: model.layers.3.self attn.v proj.weight can expose: Shape: 128 × 896 Dtype: float32 Parameters: approximately 114.7K Layer: 3 Runtime: hf local That creates a connection between the visualization and the actual model implementation. TokenPrint is also intended to be an educational tool. A component should answer more than: "What is this box?" It should explain: What does this operation compute? Why does the transformer need it? What is its input? What is its output? How does it connect to the next operation? Which parameters and tensors are involved? This makes it possible to move from a high-level explanation to the mathematical and implementation-level details without leaving the visualization. The long-term goal is not only visualization. TokenPrint is being designed as an environment for inspecting and experimenting with transformer computation. That includes areas such as: The idea is to make the internal behavior of an LLM inspectable rather than treating inference as a black box. TokenPrint is open source and built for people who want to learn, experiment, research, and contribute. There are opportunities across: GitHub: https://github.com/Sudharsanselvaraj/Token-Print https://github.com/Sudharsanselvaraj/Token-Print I'm working toward making TokenPrint much more than a visualization of one local model. Some of the directions include broader Hugging Face model support, better model capability detection, richer execution traces, remote inference, experiment workflows, and shareable transformer traces that can be explored without reproducing the original runtime. The larger idea is: What if opening an LLM did not just show you the model's architecture, but let you actually watch computation happen inside it? That's what I'm trying to build with TokenPrint. Try TokenPrint → https://tokenprint.in/ https://tokenprint.in/ Star the project or contribute → https://github.com/Sudharsanselvaraj/Token-Print https://github.com/Sudharsanselvaraj/Token-Print Explore what happens inside a transformer: follow tokens through embeddings, attention, Q/K/V projections, RoPE, MLPs, residual streams, and logits with an interactive 3D visual debugger.