Microsoft's BitNet research demonstrates that large language models can run with weights restricted to just −1, 0, or +1, eliminating multiplications and reducing memory by an order of magnitude. The …
DeepGrove's Maple-Preview, a 20B-parameter mixture-of-experts reasoning model with ternary weights, achieves 120 tokens per second on an iPhone and 218 tok/s on a base M4 Mac mini, according to the co…
A developer successfully ran an autoregressive language model on a 1975 MOS 6502 8-bit processor inside a BBC Micro, using BitNet ternary quantization to fit 13KB of model weights and 9KB of inference…
Fermion Research released Neutrino-1 8B, a 3.88 GB language model trained natively with ternary weights ({-1, 0, +1}) that achieves 763 tokens per second on an H100 with speculative decoding and 24.9 …
Bonsai 27B is a 27-billion-parameter large language model using 1-bit quantization (BitNet b1.58 architecture) that compresses to roughly 4GB, enabling it to run entirely on a high-end smartphone or l…
BitNet, a technique using ternary weights with values of -1, 0, and 1, offers the most potential for optimizing LLM inference, according to a developer's research. The approach could double the parame…
Rudrite Research compares three methods for shrinking large language models: AWQ scales salient weights, GPTQ compensates rounding with second-order math, and BitNet trains ternary weights to turn mat…