Fermion Research publishes 3.88 GB Neutrino-1 8B for local inference

Fermion Research has published Neutrino-1 8B, an 8.19-billion-parameter language model packaged in a 3.88 GB file that fits on an 8 GB GPU or 16 GB laptop, using a proprietary ternary-family format that reduces linear weight storage to one-eighth of fp16. Derived from Alibaba's Qwen3-8B, the model is designed for local inference through model-runtime co-design, with weights decoded inside matrix kernels to cut data movement per token.

Fermion Research https://www.fermionresearch.com/?ref=runtimewire has published Neutrino-1 8B https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire , packaging an 8.19-billion-parameter language model in a 3.88 GB file. Hugging Face's model card https://huggingface.co/FermionResearch/Neutrino-8B?ref=runtimewire lists the artifact as FermionResearch/Neutrino-8B; the available listing does not establish a specific launch date. Fermion says the model fits alongside its attention cache on an 8 GB GPU or a 16 GB laptop, giving developers one artifact rather than separate exports for each class of hardware. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire The release is Fermion's bet that local AI will be won through model-runtime co-design. Neutrino's weights remain packed while stored and are decoded inside the matrix kernels, cutting the amount of data that hardware must move for each generated token. Fermion built an inference engine, custom Metal kernels and a public llama.cpp fork https://github.com/fermionresearch/llama.cpp?ref=runtimewire around that format instead of treating deployment as downstream packaging work. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire The bytes are the product Neutrino-1 8B uses a proprietary ternary-family format for all 252 linear projections across its 36 transformer layers. Those 6.95 billion coded projection weights account for 2.60 GB, or 67.2%, of the file. Two untied int8 embedding tables consume another 1.24 GB, meaning almost one-third of Neutrino's footprint comes from its 151,936-token vocabulary rather than the transformer blocks. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire Fermion reports that 62.63% of the coded weights are zero, with the remainder divided almost evenly between positive and negative states. The format uses one-eighth the storage of fp16 for the linear weights, according to Fermion. Its 2.56 GB download expands losslessly to the 3,875,404,812-byte container executed by each runtime. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire That distinction matters because single-user generation frequently runs into memory-bandwidth limits. Each new token requires the processor to move much of the model through memory. Reducing the bytes moved can raise decode speed without changing the underlying memory system. Context still carries a growing cost. Neutrino's grouped-query attention uses an fp16 KV cache of 144 KiB per token, adding about 0.60 GB at 4,096 tokens and 4.83 GB at 32,768 tokens. The full 40,960-token window brings model and cache residency to just under 10 GB. fermionresearch.com https://www.fermionresearch.com/research/the-neutrino-engine/?ref=runtimewire Qwen3 is the starting point Neutrino-1 8B is derived from Alibaba's Qwen3-8B https://huggingface.co/Qwen/Qwen3-8B?ref=runtimewire , rather than a foundation model pretrained from scratch by Fermion. Qwen3 supplies the 36-layer architecture, 4,096-wide hidden state, grouped-query attention and tokenizer. Alibaba released Qwen3-8B under Apache 2.0. huggingface.co https://huggingface.co/Qwen/Qwen3-8B?ref=runtimewire The resulting artifact combines the Qwen3-derived architecture with Fermion's packed storage format and platform-specific runtimes. Fermion's public materials document the shipped container and its execution paths. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire One model, several execution paths Fermion lists three distribution routes for Neutrino. Its Python package https://pypi.org/project/fermion-research/?ref=runtimewire downloads the model and a native binary for macOS arm64 or Linux x86-64. A GGUF build runs through Fermion's llama.cpp fork with CUDA support. An MLX package uses custom Metal kernels on Apple silicon. Fermion says all three execute weights derived from the same container. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire Fermion reports 396 tokens per second for plain single-stream greedy decoding on an Nvidia H100, 30.7 tokens per second on an Nvidia L4, 33.7 tokens per second through MLX on a base M5 MacBook and 24.9 tokens per second using nine CPU threads on an Apple M5. The L4 configuration used 4.68 GiB of memory at a 4,096-token context length. These are Fermion's measurements on the shipping artifact. fermionresearch.com https://www.fermionresearch.com/models/neutrino-8b/?ref=runtimewire A crowded low-bit race Fermion enters an active effort to make ternary models practical. Microsoft Research's BitNet b1.58 2B4T https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16?ref=runtimewire demonstrated a natively trained ternary model at roughly 2 billion parameters, paired with the open-source bitnet.cpp runtime https://github.com/microsoft/BitNet?ref=runtimewire . Microsoft describes the deployment weights as 1.58-bit, with 8-bit activations. huggingface.co https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16?ref=runtimewire PrismML's Ternary Bonsai https://prismml.com/news/ternary-bonsai?ref=runtimewire family includes an 8-billion-parameter model and targets Apple devices through MLX. PrismML says Ternary Bonsai 8B is about 1.75 GB and reaches 82 tokens per second on an M4 Pro, though differences in hardware, runtime, architecture and benchmark protocol prevent a direct comparison with Fermion's M5 results. prismml.com https://prismml.com/news/ternary-bonsai?ref=runtimewire Fermion's differentiator is the delivery system around Neutrino: a Qwen3-derived 8B model, multiple kernel backends and one container intended to travel between them. Independent tests will determine whether Fermion's quality and throughput measurements hold across ordinary workloads.