Liquid AI trains 4-bit LFM2.5 checkpoints to retain roughly 97% of BF16 performance

Liquid AI, an MIT spinout, released four updated LFM2.5 checkpoints on August 19, trained with quantization-aware distillation (QAD) to retain roughly 97% of their BF16 average benchmark performance while reducing memory requirements. The 230M, 350M, 1.2B, and 2.6B parameter models are available as Q4_0 GGUF files for local inference via llama.cpp, with Q4_0 file sizes of 219 MB for the 350M model and 1.59 GB for the 2.6B model, compared to 711 MB and 5.4 GB for BF16, respectively. The company's detailed post notes QAD closed 70.6%, 73.4%, 65.5%, and 48.4% of the performance gap between standard Q4_0 and BF16 for the four models, respectively.

Liquid AI trains 4-bit LFM2.5 checkpoints to retain roughly 97% of BF16 performance The MIT spinout used quantization-aware distillation across four LFM2.5 models, then tested them on a phone, laptop, mini PC and Raspberry Pi. By RuntimeWire Staff /author/runtimewire-staff · Published Primary source: Hugging Face https://huggingface.co/blog/LiquidAI/qad Why it matters Edge AI depends on the compressed checkpoint developers can actually deploy. Liquid AI is training for that constraint directly, while its own figures show why rounded accuracy claims need scrutiny. Ramin Hasani @ramin m h https://x.com/ramin m h?ref=runtimewire and his co-founders at Liquid AI https://www.liquid.ai/?ref=runtimewire released four updated LFM2.5 checkpoints on August 19, training the models for 4-bit deployment instead of leaving compression as a final step that developers have to accept with its usual quality loss. The release on Hugging Face https://huggingface.co/blog/LiquidAI/qad?ref=runtimewire , authored by Aditya Tadimeti, Leonie Monigatti and Liquid AI, covers the 230-million, 350-million, 1.2-billion and 2.6-billion-parameter members of the LFM2.5 family. Each is available as a Q4 0 GGUF file built with quantization-aware distillation, or QAD, for local inference through llama.cpp and compatible runtimes. Quantization reduces the numerical precision used to store model weights. That cuts memory requirements and can increase inference speed, with some model quality typically lost in the conversion. Liquid AI distilled a high-precision teacher model into a student that was quantized during training. That distinction is the founders' latest attempt to make edge deployment part of model development itself. Hasani started Liquid AI in 2023 with Mathias Lechner @mlech26l https://x.com/mlech26l?ref=runtimewire , Alexander Amini @xanamini https://x.com/xanamini?ref=runtimewire and MIT CSAIL director Daniela Rus. Their research on liquid neural networks grew out of work on adaptive systems that could operate under tight computational constraints. Hasani, Liquid AI's CEO, previously worked as a machine-learning scientist at Vanguard and as a postdoctoral associate at MIT CSAIL https://www.csail.mit.edu/person/ramin-hasani?ref=runtimewire , where he studied sequential decision-making with Rus. The 97% claim needs a closer reading Liquid AI says https://huggingface.co/blog/LiquidAI/qad?ref=runtimewire the four QAD checkpoints retained 97.1%, 96.5%, 97.4% and 96.6% of their corresponding BF16 average benchmark performance. Its evaluation covered reasoning, instruction-following and tool-use tests, with results averaged over five runs. The rounded 97% figure is easy to misread. Liquid AI describes the aggregate result as recovering 97% of the BF16 average accuracy lost during quantization. Its more detailed company post https://www.liquid.ai/blog/qad?ref=runtimewire says QAD closed 70.6% of the gap between standard Q4 0 and BF16 for the 230M model, 73.4% for the 350M model, 65.5% for the 1.2B model and 48.4% for the 2.6B model. Those gap-recovery rates and the retention figures published in the Hugging Face release https://huggingface.co/blog/LiquidAI/qad?ref=runtimewire measure different things and should not be treated as interchangeable. The roughly 97% number therefore needs to be read alongside both sets of per-model results. The retention and gap-recovery figures give developers a more useful basis for deciding whether the training method justifies switching checkpoints. The memory savings are substantial enough to explain the effort. The LFM2.5-350M model card https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF?ref=runtimewire lists its Q4 0 file at 219 MB, compared with 711 MB for BF16. The 2.6B model card https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF?ref=runtimewire lists 1.59 GB at Q4 0 and 5.4 GB at BF16. Compression becomes part of the model Liquid AI tested decode throughput on a phone, laptop, mini PC and Raspberry Pi. The laptop and mini PC ran inference on their GPUs; the phone and Raspberry Pi used Arm CPUs. The Hugging Face release reports that the 230M and 350M checkpoints matched Q5 K M quality within evaluation variance while decoding 4% to 33% faster https://huggingface.co/blog/LiquidAI/qad?ref=runtimewire . Liquid AI says the 1.2B and 2.6B checkpoints matched Q4 K M quality with throughput gains of 3% to 14% https://huggingface.co/blog/LiquidAI/qad?ref=runtimewire . Liquid AI also compared two variants with Unsloth's UD-Q4 K XL builds. These remain Liquid AI's tests. Exact results will depend on runtime builds, prompts, memory configuration and hardware conditions, and outside evaluators will need to reproduce the gains. Liquid AI has at least made the checkpoints directly inspectable: developers can download LFM2.5-230M https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF?ref=runtimewire , LFM2.5-350M https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF?ref=runtimewire , LFM2.5-1.2B-Instruct https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF?ref=runtimewire and LFM2.5-2.6B https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF?ref=runtimewire and compare the QAD files with their post-training-quantized counterparts. The larger strategic point is distribution. A model designed for phones, cars and small computers gains little from impressive full-precision scores if the downloadable version loses too much capability when squeezed into available memory. Liquid AI is treating that squeeze as a training problem it owns. Hasani's edge bet gets a less glamorous layer Liquid AI emerged from MIT CSAIL with an efficiency-first pitch during an industry race dominated by larger training clusters and cloud inference. Its founders have kept returning to the same constraint: useful AI has to fit where customers want to run it. Investors have financed that thesis at considerable scale. Liquid AI announced $46.6 million in seed capital in 2023 https://www.liquid.ai/blog/new-generation-of-ai-models-from-first-principles?ref=runtimewire , followed by a $250 million Series A led by AMD Ventures https://www.liquid.ai/blog/we-raised-250m-to-scale-capable-and-efficient-general-purpose-ai?ref=runtimewire in December 2024. Liquid AI said the Series A would fund compute infrastructure, edge and on-premise inference systems, and expansion across additional model sizes and modalities. Quantization work lacks the spectacle of a larger parameter count. It is also where an edge-model maker has to prove that its architecture survives contact with actual devices. For Hasani, the QAD release turns a familiar compromise into another stage of model training, with downloadable artifacts developers can test rather than a research promise they have to take on faith.