# Exploring 1-Bit LLMs > Source: > Published: 2026-08-25 13:55:05+00:00 # Exploring 1-Bit LLMs My interest in LLMs has increasingly shifted towards smaller models, small in terms of size and their ability to run on modest local machines. For some time now, I have been using models with fewer than 40 B parameters on my machines in some quantized way. Then I came across BitNet, a project that introduced me to the concept of using just 1 bit per weight. #### Quantization [Weights are represented](https://hai.stanford.edu/ai-definitions/what-are-weights) by numbers in a model. A normal model might store each weight using 16 bits, i.e** **FP16 (half-precision floating-point). Quantization reduces that dramatically, where weights are represented using INT8 (8-bit integer) or INT4 (4-bit integer). They reduce precision but [increase inference speed](https://developer.nvidia.com/blog/int4-for-ai-inference/) and reduce hardware requirements. Most of the time, quantization is performed post-training, which reduces model accuracy. But there is INT4 QAT (4-bit Integer [Quantization-Aware Training](https://www.ibm.com/think/topics/quantization-aware-training)), which does this during training, thereby reducing the extreme accuracy drops common with standard 4-bit quantization. | Approach | Training | Inference weights | |---|---|---| | FP16 model | Full precision | FP16 | | INT8 post-training quantization | Train normally, quantize later | INT8 | | INT4 QAT | Quantization simulated during training | INT4 | | BitNet | Designed and trained around extreme low-bit weights | ternary or binary | #### 1 bit weight 1-bit goes one step further and asks: what if the weights were just one bit, and if that were done during training? With exactly 1 bit per weight, each weight can have only 2 possible values. 0 or 1. 0 represents -1 and 1 represents +1. i.e., instead of storing numbers like 0.72, -0.13, 1.24, -0.56, we will store +1, -1, +1, -1. That is at least 16 times less storage for weights theoretically. #### 1.58 bit weight In [1.58-bit weights](https://arxiv.org/abs/2504.12285v2) (ternary bit weights), there are three possible values: {-1, 0, +1}. Theoretically, we need log2(3) bits of information to distinguish between 3 possible states : 2x=3. i.e. log2(3)≈1.585. Hence the name. For our interpretation and understanding: - 0: the input is effectively ignored for that connection. i.e., no influence at all, which also gives you built-in sparsity. - +1: the input contributes positively to the output. i.e., an increase in the input pushes the weighted sum up. This is “same-direction” influence. - -1: the input contributes negatively. i.e., an increase in the input pushes the weighted sum down. This is “opposite-direction” influence. #### 1.125-bit weight 1.125-bit applies different logic for compression and calculation. One can’t literally allocate 1.125 bits per weight. The way this works is that the average bit allocation is used when they are packed. For example, it takes 9 bits to store 8 weights; thus, the average number of bits per weight is 1.125. One important distinction here is that bits per weight doesn’t necessarily tell you how many values each individual weight can take. **1.58 has a clean theoretical connection** to three states, while **1.125 bpw** usually describes the overall encoding/storage scheme. The table below should make a bit clear | Format | Possible basic values | Approx storage / 1B weights | |---|---|---| | FP16 | ~65,536 | 2 GB | | INT8 | 256 | 1 GB | | INT4 | 16 | 500 MB | | Ternary | 3 | ~198 MB at theoretical 1.58 bpw | | 1.125-bit | special packed encoding | ~141 MB | | Binary | 2 | 125 MB | #### BitNet b1.58 2B4T This is the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale, developed by Microsoft Research. You can find the [model on the hugging face](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T), it needs a specific inference code called [bitnet.cpp](https://github.com/microsoft/BitNet). #### Bonsai models [Bonsai 1-bit models](https://huggingface.co/collections/prism-ml/bonsai) are probably the first set of production-ready binary/ternary models. They have [1.7B](https://huggingface.co/prism-ml/Bonsai-1.7B-gguf), [4B](https://huggingface.co/prism-ml/Bonsai-4B-gguf), [8B](https://huggingface.co/prism-ml/Bonsai-8B-gguf), and [27B](https://huggingface.co/prism-ml/Bonsai-27B-gguf) parameter models. I have been running the 8B-parameter version locally. My experience with it has been positive. I have explored it for text summaries, code explanations, etc. I will try to install the 27B version and do some coding to see how it goes. ``` # Install llama curl -LsSf https://llama.app/install.sh | sh # Serve the model using OpenAI-compatible API and a web ui llama serve -hf prism-ml/Bonsai-1.7B-gguf:Q1_0 # or 8B parameter version llama serve -hf prism-ml/Bonsai-8B-gguf:Q1_0 ``` Note: Yes, I have moved away from Ollama and now mostly use [llama.cpp](https://llama.app/). Also I sometimes [Jan.ai](https://www.jan.ai/) along with [OpenWebUI](https://openwebui.com/)