{"slug": "thriftattention-selective-mixed-precision-for-long-context-fp4-attention", "title": "ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention", "summary": "Researchers have developed ThriftAttention, a mixed-precision attention algorithm that selectively computes only 5% of query-key blocks in FP16 precision while processing the remaining 95% in FP4, recovering 89.1% of the performance gap between FP4 and FP16 attention on long-context benchmarks. The method addresses quality degradation in long-context workloads by identifying and preserving high-importance token interactions at full precision, achieving near-FP16 quality with FP4-level inference efficiency. ThriftAttention's advantage grows with sequence length, mitigating the systematic quality loss that previously limited 4-bit attention in extended contexts.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 21 May 2026]\n\n# Title:ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention\n\n[View PDF](/pdf/2605.23081)\n\n[HTML (experimental)](https://arxiv.org/html/2605.23081v1)\n\nAbstract:Efficient attention algorithms are critical to mitigate the quadratic cost of attention in long-context workloads. Prior work utilises block-scaled quantisation techniques on Blackwell GPUs to move attention computation to 4-bit precision to accelerate inference. However, these techniques result in significant quality degradation in long-context settings. We show that the output impact of quantisation error is highly non-uniform and increases with the importance of each query-key interaction, concentrating functionally relevant error in a small number of attention blocks that contain the most important tokens. We propose ThriftAttention, a low-bit attention variant that delivers near-FP16 long-context quality at FP4 inference efficiency. This approach proceeds in two stages. First, a heuristic rapidly selects a small number of important query-key block pairs for FP16 precision. Second, the selected blocks are computed in FP16 and the remaining blocks in FP4, with both paths merged via online softmax into a single output. We demonstrate across long-context benchmarks and model families that by computing only 5% of query-key blocks in FP16, ThriftAttention recovers on average 89.1% of the FP4-to-FP16 performance gap. We show ThriftAttention's advantage grows with sequence length, mitigating the systematic FP4 quality degradation observed at longer contexts. The code is available at[this https URL].\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/thriftattention-selective-mixed-precision-for-long-context-fp4-attention", "canonical_source": "https://arxiv.org/abs/2605.23081", "published_at": "2026-05-26 06:22:05+00:00", "updated_at": "2026-05-26 06:38:54.135053+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "ai-infrastructure", "ai-chips"], "entities": ["ThriftAttention", "Blackwell GPUs", "FP4", "FP16"], "alternates": {"html": "https://wpnews.pro/news/thriftattention-selective-mixed-precision-for-long-context-fp4-attention", "markdown": "https://wpnews.pro/news/thriftattention-selective-mixed-precision-for-long-context-fp4-attention.md", "text": "https://wpnews.pro/news/thriftattention-selective-mixed-precision-for-long-context-fp4-attention.txt", "jsonld": "https://wpnews.pro/news/thriftattention-selective-mixed-precision-for-long-context-fp4-attention.jsonld"}}