{"slug": "lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation", "title": "LFM2.5 Q4\\_0 Checkpoints from Quantization-Aware Distillation", "summary": "Liquid AI released Q4_0 checkpoints for its LFM2.5 language models trained with quantization-aware distillation (QAD), recovering 96.5% to 97.4% of the accuracy lost to quantization across four models. The QAD checkpoints match the quality of higher-bitrate Q5_K_M or Q4_K_M GGUFs at 3-33% higher decode throughput on edge hardware, and are available on Hugging Face for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.", "body_md": "Text Generation • 1B • Updated • 239k • 208\n\n# LFM2.5 Q4_0 Checkpoints from Quantization-Aware Distillation\n\n[Team Article](/blog)\n\n**Trained with Quantization-Aware Distillation (QAD)**: a high-precision teacher model is distilled into a quantized student model** Same memory and speed as native Q4_0:**They keep the low memory footprint and high throughput of Q4_0 GGUFs** Recovery:**97% of their BF16 average accuracy lost to quantization is recovered\n\n## Benchmark results\n\nFor all four models, we compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The BF16 GGUF serves as the in-format ceiling. We also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B. We report the mean across five repeats.\n\nAcross all four models, **QAD substantially improves the Q4_0 checkpoint.** The QAD checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance.\n\n## Speed and size on real edge hardware\n\nWe measure decode throughput for the four models LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B across four targets: MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. MacBook Pro and NucBox use GPU inference, while Samsung and Raspberry Pi use Arm CPU inference. BF16 and F16 are shown as full-precision references where profiled.\n\nThe 230M and 350M QAD Q4_0 checkpoints match Q5_K_M quality within evaluation variance at a 4-33% higher decode throughput. The 1.2B and 2.6B QAD Q4_0 checkpoints match Q4_K_M quality at a 3-14% higher throughput. The QAD Q4_0 checkpoints also match Unsloth's UD-Q4_K_XL (where applicable, for the 230M and 1.2B), a strong external post-training quantization checkpoint.\n\n## How to use QAD GGUFs\n\nUse the files with llama.cpp or any runtime that supports GGUF Q4_0 artifacts.\n\n```\nllama-cli -hf LiquidAI/LFM2.5-350M \\\n  --hf-file LFM2.5-350M-QAD-Q4_0.gguf \\\n  -p \"What is C. elegans?\"\n```\n\n## Get Started with QAD GGUFs\n\nThe QAD GGUFs are available on Hugging Face today: [LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF), [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF), [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF), and [LFM2.5-2.6B](https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF).\n\nWe can't wait to see what you build.\n\n## Citation\n\nFor citations, please use the following reference or BibTeX:\n\n```\nLiquid AI, \"LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment\", Liquid AI Blog, Aug 2026.\n```\n\nOr use the BibTeX citation\n\n```\n@article{liquidAI2026Q40,\n  author = {Liquid AI},\n  title = {LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment},\n  journal = {Liquid AI Blog},\n  year = {2026},\n  note = {www.liquid.ai/blog/qad},\n}\n```\n\n", "url": "https://wpnews.pro/news/lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation", "canonical_source": "https://huggingface.co/blog/LiquidAI/qad", "published_at": "2026-08-19 13:48:49+00:00", "updated_at": "2026-08-19 14:14:00.474848+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "ai-products"], "entities": ["Liquid AI", "LFM2.5-230M", "LFM2.5-350M", "LFM2.5-1.2B-Instruct", "LFM2.5-2.6B", "Hugging Face", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation", "markdown": "https://wpnews.pro/news/lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation.md", "text": "https://wpnews.pro/news/lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation.txt", "jsonld": "https://wpnews.pro/news/lfm2-5-q4-0-checkpoints-from-quantization-aware-distillation.jsonld"}}