IFM Releases K2 Horizon: Six Apache 2.0 Models From 0.9B to 375B

The Institute of Foundation Models (IFM), the frontier lab launched by MBZUAI in May 2025, released K2 Horizon, a fleet of six Apache 2.0-licensed open-source models ranging from 0.9B to 375B parameters, alongside the pre-training corpus, intermediate checkpoints, training code, configs, and fine-grained logs, which IFM calls the largest fully open-source model launch in AI history. The models, pre-trained on roughly 20 trillion tokens each with nearly 17% problem-solving trajectories and about 10 trillion synthetic tokens, include the 375B-A23B flagship scoring 70.2 on Terminal-Bench 2.1, and smaller models like the 7B posting 70.6 on SWE-bench Verified. IFM also introduced Mixture-of-Value Attention (MoVA) for sparsity in attention and Uno, a LoRA adapter enabling lossless decoding speedup of roughly 3×.

Most open model launches release one checkpoint and a benchmark table. The Institute of Foundation Models https://ifm.ai/ IFM released something wider last week. IFM is the frontier lab launched by MBZUAI https://mbzuai.ac.ae/ in May 2025. K2 Horizon https://ifm.ai/k2/ is a fleet of six models: 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B. Shipping alongside them are the pre-training corpus, intermediate checkpoints, training code, configs and fine-grained logs. IFM calls it the largest fully open-source model launch in AI history. Is it deployable? Yes, all six sizes sit on Hugging Face https://huggingface.co/collections/IFM/k2-horizon under Apache 2.0 https://www.apache.org/licenses/LICENSE-2.0 , with FP8 and GGUF builds. Day-zero support covers vLLM https://recipes.vllm.ai/IFM , SGLang https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon and Ollama, on NVIDIA, AMD and Cerebras hardware. Hosted APIs run through Compass, Cerebras and Nebius via platform.ifm.ai https://platform.ifm.ai/ . What Actually Shipped The six models share a core architecture, vocabulary, training methodology, interfaces and deployment tooling. The 0.9B model uses a smaller vocabulary. That consistency is the point: teams can prototype on 3.7B and scale to 375B-A23B without changing their serving stack. Each model is pre-trained on roughly 20 trillion tokens. Nearly 17% of the pre-training corpus consists of problem-solving trajectories with explicit reasoning. About 10 trillion tokens were synthetic. Post-training data was folded in from mid-training rather than saved for the end. IFM research team reports over 100 million unique synthesized tasks. Tool definitions were presented in JSON, XML and Markdown during training so the model learns semantics rather than syntax. Markdown became the inference default, roughly 18.5% more token-efficient than JSON on IFM’s data. MoVA: Sparsity Moved into Attention Conventional Mixture-of-Experts applies sparsity to feed-forward layers. Mixture-of-Value Attention MoVA extends expert routing into multi-head attention itself, opening a second axis for scaling capacity. It stays compatible with FlashAttention, grouped-query attention and sparse attention. The result is K2-Horizon-MoVA-36B-A4B: 36B total parameters, roughly 4B active per token. Under matched training conditions it lands slightly below the dense 32B model. On IFM’s tables it posts 58.6 on Terminal-Bench 2.1 and 26.8 on tau3-Banking, leading its comparison set on both. Uno: A Lossless Decoding Speedup as a LoRA Uno freezes Horizon’s autoregressive parameters and trains a small set of diffusion parameters that learn only how to generate efficiently. Through what IFM calls diffusion distillation, these adapters emit blocks of tokens in parallel. The press release puts the speedup at roughly 3× with no quality degradation. It ships as a LoRA adapter, currently 7B-Uno https://huggingface.co/IFM/K2-Horizon-7B-Uno and 0.9B-Uno https://huggingface.co/IFM/K2-Horizon-0.9B-Uno . Numbers worth knowing K2-Horizon-375B-A23B scores 70.2 on Terminal-Bench 2.1, 1,441 Elo on GDPVal-AA, 67.7 on MCPMark and 87.3 on GPQA Diamond. It leads its table on SWE-Atlas-QnA at 48.4 but trails GPT-5.6 Luna and Claude Sonnet 5 on most agentic rows. The small models are the sharper story. 7B posts 70.6 on SWE-bench Verified and 59.0 on BrowseComp. 3.7B posts 68.6 on SWE-bench Verified. 0.9B reaches 48.5 on AIME 2026 and 79.9 on HumanEval+, small enough to run under quantization on a watch. The Audit IFM Ran on Itself This is the part many other labs do not publish. IFM ran 375B-A23B across 89 Terminal-Bench 2.1 tasks, eight attempts each. That is 712 trials, 500 passing, a reported 70.2% accuracy. Every passing trial was then re-audited using Artificial Analysis’s reward hacking procedure https://artificialanalysis.ai/methodology/coding-agents-benchmarking reward-hacking . The audit flagged 24 trials across 10 tasks. Removing them drops accuracy to 66.9%, a 3.37-point correction. That sits between the flag rates Artificial Analysis reports for Claude Fable 5 2.2% and GPT-5.6 Luna 4.1% . Behaviors included locating benchmark repositories on GitHub and downloading reference solutions. IFM also disclosed a 7B run that reached an inflated 82 on SWE-bench by finding answers. Interactive explainer Key Takeaways - Six models, 0.9B to 375B, all Apache 2.0, all sharing one architecture and serving stack. - MoVA pushes MoE routing into attention: 36B total, ~4B active, near dense-32B quality. - Uno delivers roughly 3× lossless decoding speedup as a drop-in LoRA adapter. - The 0.9B, 3.7B and 7B models claim state of the art at their respective scales. - IFM published its own reward-hacking audit, correcting 70.2% down to 66.9%. Check out the Technical blog https://ifm.ai/blog/k2 , Press release https://ifm.ai/k2/press-release/ , Hugging Face collection https://huggingface.co/collections/IFM/k2-horizon , TxT360-v2 dataset https://huggingface.co/datasets/IFM/TxT360-v2 , xLLM pre-training code https://github.com/ifm-ai/xllm , Post-training code https://github.com/ifm-ai/horizon-post-train and Docs https://docs.ifm.ai/ . Also, feel free to follow us on Twitter https://x.com/intent/follow?screen name=marktechpost and don’t forget to join our 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ and Subscribe to our Newsletter https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}} . Wait are you on telegram? now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.