Latent Sparse Attention (LSA): The 72.4x and 22.7x Speedups, and What They Compare Against

A paper titled Latent Sparse Attention (arXiv 2610.42147), submitted October 30, 2026 by Ales Kubicek, Torsten Hoefler and eight co-authors, reports that its D-LSA variant speeds up a single attention layer by 22.7x in prefill and 4.6x in decode on GLM-5.3-Flash versus that model's own sparse attention, and by 72.4x in prefill and 42.5x in decode on Qwen3.8-27B versus full attention, at 10 million tokens on NVIDIA H200 GPUs. The method clusters keys in a learned latent space so queries score cluster centroids rather than every key, leaving the attention kernel unchanged, and the authors say it keeps similar accuracy across a variety of tasks. The code is released under Apache 2.0 as an SGLang plugin, and no independent reproduction of the numbers was found.

A new paper called Latent Sparse Attention reports that it can make one attention layer up to 72.4x faster at 10 million tokens of context. That number is in the abstract of arXiv 2610.42147 https://arxiv.org/abs/2610.42147?ref=explainx , submitted on October 30, 2026. It is real, but it is easy to misread. It is a single-layer prefill speedup on one model, compared with full attention. A second number, 22.7x, compares a different model against its own sparse attention. This post explains the idea, separates the two baselines, and lists what you would need to run it. We read the abstract and the project README https://github.com/Prefrontal-AI/latent-sparse-attention?ref=explainx . We did not run the code or reproduce the numbers. TL;DR: what the paper claims | Question | Answer | |---|---| | What is LSA? | An attention method that clusters keys in a learned latent space so queries score clusters, not every key | | Who wrote it? | Ales Kubicek, Andrea Caciolai, Liangzu Peng, Sukrit Sumant, Benjamin Hoover, Fraser Greenlee, Ruairidh M. Battleday, Radim Urban, Torsten Hoefler and James C.R. Whittington | | Headline result | D-LSA: 22.7x prefill and 4.6x decode on GLM-5.3-Flash, against its own sparse attention; 72.4x prefill and 42.5x decode on Qwen3.8-27B, against full attention. One attention layer, 10M tokens | | Accuracy | The authors say it keeps similar accuracy across a variety of tasks | | Native variant | N-LSA matches full-attention accuracy and trains 1.6x faster per layer at 128k tokens | | Code | Apache 2.0, an SGLang plugin; the repo showed 32 stars when we checked | | Hardware | Results measured on NVIDIA H200 GPUs, per the README | | Independent check | None found | Why does ultra-long context need a new kind of attention? The abstract opens with agents. It says agentic workflows are shifting toward long-horizon tasks, such as open-ended scientific work, where models benefit from long context windows. Above about one million tokens, it says, full attention is "prohibitively expensive," because every token compares itself with every earlier token. Cost grows with the square of context length. For a primer on that mechanism, see our transformer architecture guide https://explainx.ai/blog/what-is-transformer-architecture-attention-llm-guide-2026 . Sparse attention is the common fix. A small component called an indexer picks a subset of keys for each query, and attention runs only on those. The authors say existing indexing either still has quadratic cost in context size, or loses accuracy at ultra-long scale. The indexer becomes the bottleneck: at 10M tokens, deciding which keys to read can cost more than reading them. How does LSA work? LSA changes how the indexer searches. 1. Cluster the keys. Keys are pooled and clustered into buckets in a learned latent space, organized as a hierarchy. 2. Score clusters, not keys. A query scores the bucket centroids first, then drills into the selected buckets. 3. Attend to a small set. The query attends to the selected long-range keys plus the attention sink and a local window. The README is explicit that "the attention kernel is unchanged. LSA only decides which keys it reads." That is why it can drop into an existing serving stack. It also means any gain comes from cheaper key selection. The paper offers two variants: - D-LSA Distilled applies to a pre-trained model. You distill the model's indexer into the clustered space. The repo lists flat search, tree search and a training-free option that uses a PCA map. - N-LSA Native is trained from scratch on next-token loss alone. The README compares it with full attention, sliding-window attention, NSA and PISA baselines. What do 22.7x and 72.4x actually compare? The abstract packs both numbers into one sentence: on GLM-5.3-Flash Qwen3.8-27B at 10M tokens, D-LSA speeds up one attention layer by 22.7x 72.4x in prefill and 4.6x 42.5x in decode over its original sparse attention full attention . The parentheses pair each figure with a different model and baseline. The README states it more clearly. | Model | Baseline | Prefill speedup | Decode speedup | |---|---|---|---| | GLM-5.3-Flash | The model's own sparse attention | 22.7x | 4.6x | | Qwen3.8-27B | Full attention | 72.4x | 42.5x | Three cautions apply. - Different baselines. The model that gets 72.4x is compared with full attention, which is the slowest option at 10M tokens. A large gap against full attention is expected from any sparse method. The fairer test of LSA's indexer is the 22.7x figure, which compares against an existing sparse design. The two numbers are not a range for one model. - One layer, not a whole model. Both figures are for "one attention layer." A full model also spends time in feed-forward layers, communication and memory movement. End-to-end gains will be smaller. The paper's abstract gives no end-to-end figure. - Author-run, at a very long context. The 10M-token setting is where the quadratic cost is worst, so it shows the method at its best. The README says the paper's results came from H200 GPUs, and runs of that size need context parallelism. None of this makes the result false. It means the headline needs its footnotes. For context on how other labs frame similar claims, see our look at Zyphra's hierarchical attention https://explainx.ai/blog/zyphra-zha-hierarchical-attention-subquadratic-tree-indexer-qwen3-6-35b-ruler-modeled-speedups-2026 , which also uses a tree-style indexer and reports modeled speedups, and our report on SubQ's sparse attention at 12M tokens https://explainx.ai/blog/subq-ssa-sparse-attention-12m-context-2026 . Prefill, decode and training: why three speedups matter differently The paper reports speedups for three phases, and each matters to a different reader. Prefill is the pass where the model reads your whole prompt before it writes anything. At 10M tokens this is the wait before the first output token, and it is where attention cost is largest. A big prefill gain helps anyone who loads a huge codebase, archive or conversation history into one request. Decode is the token-by-token generation after that. Each new token attends back over the whole history, so cost per token grows with context. The 4.6x and 42.5x decode figures describe how much cheaper each generated token gets at 10M tokens, again for one layer. Training is separate. N-LSA's 1.6x per-layer gain is at 128k tokens, a far shorter context than the 10M used for the headline inference numbers. That is a modest gain, and the claim that matters there is accuracy parity with full attention. Mixing these up is the easiest way to overstate the result. A reader who sees 72.4x and assumes training or serving costs fall 72x is wrong on three counts: it is prefill only, one layer only, and against full attention only. A fair summary is that the indexer search, which the authors describe as the bottleneck at this scale, gets much cheaper, and that the gain should shrink once the rest of the model is included. What about accuracy? The abstract says D-LSA keeps "similar accuracy across a variety of tasks" and N-LSA "matches the accuracy of full attention." Those are summary claims. The abstract does not list the tasks or margins, and we did not read the full paper's tables. Treat "similar" as a claim to check against the paper's evaluation section before you rely on it for retrieval or reasoning over long documents. Can I run LSA? The code is open under Apache 2.0, and the setup is a research one. - Install. The README requires Python 3.12 and uv. uv sync --locked --extra evaluation --extra training installs the tools, and the training extra builds causal-conv1d , which needs the CUDA toolkit. - Serving. LSA is an SGLang plugin. It runs in the authors' SGLang fork, tagged lsa-v0.1.0 , which adds context-parallel and long-context support. A Docker image and an enroot path for Slurm are provided, and B300 Blackwell nodes use their own image tag. - Distilled map. D-LSA needs a distilled map for each model. You train one or use one the authors provide. - Required flag. --disable-prefill-cuda-graph is required. LSA selects keys for a prefill on the host, which a CUDA graph cannot capture, and the server crashes at startup without the flag. Decode is still captured. - Check it is active. The server log prints armed when the plugin is on. Without that line you are running the checkpoint's own attention. The README example uses Qwen/Qwen3.8-27b. For background on that model, see our Qwen 3.8 27B open-weight write-up https://explainx.ai/blog/qwen-3-8-27b-open-weight-model-claude-opus-comparison-august-2026 and the GLM-5.3-Flash launch coverage https://explainx.ai/blog/glm-5-3-flash-ox-alpha-official-launch-august-2026 . What this means for builders - If you serve short contexts, nothing changes. The gains are at very long contexts, where indexing dominates. - If you run long-context agents, watch it. It targets the cost of reading huge histories, which is the problem for long-horizon agents. - Wait for replication. One group's single-layer measurements at 10M tokens are a start. Look for end-to-end throughput, memory use and long-context accuracy from other teams. - Mind the engineering cost. A distilled map per model, a forked serving stack and disabled prefill graphs are real costs for a production system. - Compare like with like. When someone quotes a speedup, ask for the baseline, the context length, whether it is one layer or the whole model, and the hardware. What is still unknown? - End-to-end speedup for a full model and serving workload. - Accuracy detail by task, since the abstract is a summary. - Independent replication. We found none. - Behavior at other scales. The paper's authors project nothing in the abstract, and we make no projection either. - Upstreaming. The README describes a fork. Whether SGLang adopts it is not stated. Related reading Details are accurate as of November 3, 2026 and come from the arXiv abstract and the repository README. We did not run the code or read every table in the paper.