LittleBit: Ultra Low-Bit Quantization via Latent Factorization Samsung Labs researchers released LittleBit, a quantization framework that compresses large language models to as low as 0.1 bits per weight (BPW), a roughly 31x memory reduction that shrinks Llama2-13B to under 0.9 GB. The method represents weights via low-rank latent matrix factorization and binarizes the factors, adding multi-scale compensation across row, column and latent dimensions plus Dual Sign-Value-Independent Decomposition (Dual-SVID) initialization and Residual Compensation. At 0.1 BPW, LittleBit outperforms leading techniques operating at 0.7 BPW on Llama2-7B and offers a potential 11.6x inference speedup versus FP16, with code published on GitHub. Computer Science Machine Learning Submitted on 30 May 2025 v1 https://arxiv.org/abs/2506.13771v1 , last revised 5 Feb 2026 this version, v5 Title:LittleBit: Ultra Low-Bit Quantization via Latent Factorization View PDF https://arxiv.org/pdf/2506.13771 HTML experimental https://arxiv.org/html/2506.13771v5 Abstract:The deployment of large language models LLMs is frequently hindered by prohibitive memory and computational requirements. While quantization mitigates these bottlenecks, maintaining model fidelity in the sub-1-bit regime remains a persistent challenge. In this paper, we introduce LittleBit, a novel framework for extreme LLM compression. We target quantization rates as low as $0.1$ bits per weight BPW , achieving a memory reduction of approximately $31\times$, which effectively compresses Llama2-13B to under $0.9$ GB. We represent weights via low-rank latent matrix factorization and subsequently binarize the resulting factors. To counteract the information loss inherent to such drastic precision reduction, we integrate a multi-scale compensation mechanism that learns importance parameters across row, column, and latent dimensions. Two primary contributions enable effective training: Dual Sign-Value-Independent Decomposition Dual-SVID for quantization-aware training QAT initialization, and Residual Compensation to minimize approximation errors. Extensive experiments confirm the superiority of LittleBit in the sub-1-bit domain; for instance, our method at $0.1$ BPW surpasses the performance of leading techniques operating at $0.7$ BPW on Llama2-7B. We establish a new size-performance trade-off -- unlocking a potential $11.6\times$ inference speedup relative to FP16 -- and render powerful LLMs practical for resource-constrained environments. Our code is available at this https URL https://github.com/SamsungLabs/LittleBit . Submission history From: Dongkyu Kim view email https://arxiv.org/show-email/2af7be41/2506.13771 Fri, 30 May 2025 06:43:03 UTC 4,465 KB \ v1\ https://arxiv.org/abs/2506.13771v1 Tue, 28 Oct 2025 10:57:14 UTC 4,388 KB \ v2\ https://arxiv.org/abs/2506.13771v2 Thu, 4 Dec 2025 22:56:58 UTC 4,380 KB \ v3\ https://arxiv.org/abs/2506.13771v3 Thu, 15 Jan 2026 10:46:32 UTC 4,564 KB \ v4\ https://arxiv.org/abs/2506.13771v4 v5 Thu, 5 Feb 2026 01:59:26 UTC 4,587 KB Additional Features Current browse context: cs.LG References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender IArxiv Recommender What is IArxiv? https://iarxiv.org/about arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .