Fermion Research released Neutrino-1 8B, a 3.88 GB language model trained natively with ternary weights ({-1, 0, +1}) that achieves 763 tokens per second on an H100 with speculative decoding and 24.9 …
Bonsai 27B is a 27-billion-parameter large language model using 1-bit quantization (BitNet b1.58 architecture) that compresses to roughly 4GB, enabling it to run entirely on a high-end smartphone or l…
BitNet, a technique using ternary weights with values of -1, 0, and 1, offers the most potential for optimizing LLM inference, according to a developer's research. The approach could double the parame…
Rudrite Research compares three methods for shrinking large language models: AWQ scales salient weights, GPTQ compensates rounding with second-order math, and BitNet trains ternary weights to turn mat…