# Kimi-Linear-48B on Tenstorrent Wormhole > Source: > Published: 2026-08-17 14:00:00+00:00 # Kimi-Linear-48B on Tenstorrent Wormhole Veso AI open-sources the fused KDA, MoE, and MLA kernels that took Kimi-Linear-48B from not starting at all to a 16K-context server on four n300 cards. Veso AI has released **tenstorrent-wormhole-kimi-linear**, the kernels and correctness gates that run Kimi-Linear-48B on Tenstorrent Wormhole hardware. The layer programs are written in Python through `ttnn.generic_op` . There is no C++ rebuild in the loop. The model did not run when we started. It now serves 16,384 tokens of context at 13.5 tokens per second on four n300 cards. This post records what we measured, how we drove the work, and what is still wrong. ## Release facts | Item | Value | |---|---| | Repository | | `kernels/` , `tools/` , `probes/` , `engines/` ## The first condition Kimi-Linear-48B did not start. There were two causes. - The HuggingFace files do not include a `tokenizer.json` file. The daemon cannot start without this file. - The chat template sends the token `<|im_start|>` . This token is not in the Kimi vocabulary. We built the tokenizer and corrected the template. Then the model started. The tokenizer builder is in `tools/` . ## What we measured | Stage | Tokens per second | Context | Status | |---|---|---|---| | Eager ttnn | 1.44 | 256 | Measured | | Traced step | 10.55 | 256 | Measured | | Fused L1 kernels | 10.86 | 256 | Measured | | Paged flash MLA | 13.5 | 16,384 | Measured | Every number is taken through a real HTTP client, not a test harness. The reason for that rule is in the next section. Four changes produced the result. **Trace capture.** All 27 layers are recorded as one trace. This removed the host from the decode loop and is the single largest gain in the table, 1.44 to 10.55.**Fused layer programs.** One program each for KDA, MoE, and MLA, written in Python through`ttnn.generic_op` .**Batch 32.** The FPU computes 32 tile rows for every operation. A batch of one discards 31 of them. The serving path uses all 32, so 32 conversations share one device step.**Paged flash MLA.** We deleted our own MLA kernel and called Tenstorrent’s`paged_flash_multi_latent_attention_decode` on the 576 wide compressed latent. ## The constraint moved The context ceiling was 256 tokens. The cause was not DRAM. It was L1. Our attention kernel put the mask, the scores, and the softmax for the whole padded cache into L1 at the same time. That footprint grows with sequence length. Each core has 1.46 MB of L1. The ceiling followed from the arithmetic, not from the memory on the card. `probes/l1_budget.py` computes this budget. The paged latent cache reads 21 times fewer KV bytes per step. Context went from 256 to 16,384 tokens, a factor of 64, and decode went from 9.6 to 13.5 tokens per second at the same time. Memory per conversation fell from 5.32 GiB to 0.197 GiB, a factor of 27. The constraint did not disappear. It moved. DRAM now sets the maximum context, which is the expected place for it to sit. ## The method These rules let an agent work on bare metal safely. They are the transferable part of this work. - Use an fp32 torch model as the oracle. Compare every kernel against it. - Run a cheap gate before an expensive one. A one-layer gate resolves in seconds what an 8-layer gate takes 6 minutes and a full load takes 22 minutes to show. - Run an adversarial static audit in parallel with hardware work. The audit and the hardware found different faults. - A test that passes is not proof. Examine the number. - Write the plan to a file. The file holds the plan when the agent loses context. - Stop after two failures and return to the last good configuration. We found 15 faults. Three produced no error message at all. - A reshape operation that is correct at a batch of one only. - A control flag that used the slow path and reported success. - An environment variable that never reached the container, which made a passing test meaningless. ## What is not working The repository states these limits, and so does this post. | Item | Status | |---|---| | Chunked prefill | Built, measured 28 times faster, found wrong. It is disabled. | | Prefill latency | 74 ms per token on the eager path | | Prefix cache | Not implemented. Cache hit rate is 0 on every agent turn. | | Traceable prefill | Blocked. Eager prefill and a captured decode trace cannot operate together. A device tensor for `chunk_start_idx` on `chunked_flash_mla_prefill` would remove this. | ## The K3 ramp Kimi K3 reports `model_type: kimi_linear` . It is the same architecture at larger scale: 93 layers against 27, 896 experts against 256, and the same 3 to 1 ratio of KDA to full attention layers. The parts that matter to this repository are identical. `kv_lora_rank` is 512, `qk_rope_head_dim` is 64, and the MLA latent is 576 wide in both models. The paged latent path in this release applies to K3 without change. So does the tokenizer we built. The gap is memory, not architecture. K3 is about 1.56 TB in bf16. At bfp4 that is roughly 438 GB, and at bfp8 roughly 829 GB. A Wormhole Galaxy is 32 chips and about 384 GB. The arithmetic gives 2 Galaxies at bfp4 and 3 to 4 at bfp8, before cache and activations. This matches Tenstorrent’s own published DeepSeek deployment targets. We have never run on a Galaxy. The sizing above is arithmetic. It is not a measurement, and we make no throughput claim for K3. ## Scope - Every throughput figure is for Kimi-Linear-48B on four n300 cards, measured through an HTTP client. - We did not test other Wormhole configurations, Blackhole hardware, or any Galaxy system. - We did not test the vision path. K3 declares one. This engine has no vision support. - Chunked prefill numbers are excluded from the results table because the path is wrong and disabled. - Bit-exact agreement with the torch oracle was evaluated as a cost and not pursued to completion. ## Reproduce ``` git clone https://github.com/Veso-AI-Open-Source/tenstorrent-wormhole-kimi-linear python tools/gate_mla_layer.py # one-layer oracle gate python probes/l1_budget.py # the L1 arithmetic behind the 256 token ceiling ./engines/tt_kimi.sh # launch the server ``` Related reading: our [Kimi K3 forensic analysis](/blog/kimi-k3-forensic-analysis/) audits the model claims, and [the agentic harness architecture](/blog/the-agentic-harness-architecture/) covers the control plane pattern the gates above follow. *Veso AI runs its R&D in the open. Questions about the kernels or the hardware? Get in touch.*