{"slug": "run-fast-private-agents-on-device-with-lfm2-5-24b-a2b", "title": "Run fast, private agents on-device with LFM2.5-24B-A2B", "summary": "LiquidAI released LFM2.5-24B-A2B, a 24B-parameter reasoning model with 2B active parameters per token, built for on-device agent workloads. The model scores 48.09 on LiveClaw versus 38.19 for Qwen3.6-35B-A3B, runs at 184 tok/s on an Apple M5 Max and up to 21K output tok/s on a single NVIDIA B300 GPU, and expands context from 32K to 131K tokens with a vocabulary doubled from 65K to 128K via in-place tokenizer expansion. Pre-training scaled from ~17T to ~33T tokens, with mid-training adding 2T tokens at 32K context and 300B tokens at 131K context.", "body_md": "[Text Generation •  24B • Updated   •  2.88k  •  355](https://huggingface.co/LiquidAI/LFM2-24B-A2B)  \n\n# \n\t\tRun fast, private agents on-device with LFM2.5-24B-A2B\n\t\n\n [Team Article](https://huggingface.co/blog)\n\n[is a 24B-parameter reasoning model with just 2B active parameters, built for fast, private agents on-device. It thinks before it answers, calls tools, and works through multi-step tasks inside popular agent harnesses. Because only 2B parameters are active per token, each reasoning token is cheap, so you can run agents continuously on your own hardware.](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B)\n\n**LFM2.5-24B-A2B**\n- **Best-in-class agent:** Competitive with Qwen3.6-35B-A3B, a model with ~50% more total and active parameters, on agentic and tool-use tasks, scoring 48.09 on LiveClaw versus 38.19.\n- **Reasoning at MoE speed:** Always thinks before it answers. With only 2B active parameters per token, each reasoning token is cheap.\n- **Efficient inference:** 184 tok/s on an Apple M5 Max and up to 21K output tok/s on a single NVIDIA B300 GPU.\n\n## \n\t\tHow we built a reasoning agent with 2B active parameters\n\t\n\nLFM2.5-24B-A2B builds on [LFM2-24B-A2B](https://huggingface.co/LiquidAI/LFM2-24B-A2B) and keeps its architecture: 40 layers, 64 experts per MoE block with top-4 routing, and the same hybrid of gated short convolutions and GQA. The context window grows from 32K to 131K tokens, and an [in-place tokenizer expansion](https://www.liquid.ai/blog/tokenizer-expansion) doubles the vocabulary from 65K to 128K tokens for better non-Latin tokenization. Unlike its predecessor, it is a reasoning-only model that produces an explicit chain of thought before its final answer.\n\nPre-training was scaled up from ~17T to ~33T tokens. Mid-training then adds 2T tokens at 32K context, weighted toward reasoning, math, code, instruction following, and multilingual text, and 300B tokens at 131K context, weighted toward long-context data. Tool calls and multi-turn trajectories appear throughout both phases. Post-training turns the base model into an agent in five stages:\n\n1. **Supervised fine-tuning (SFT)**\n2. **Teacher specialization**\n3. **Multi-domain on-policy distillation (MOPD), non-agentic:** distill the specialist teachers on non-agentic tasks.\n4. **MOPD, agentic:** a second distillation stage focused on agentic tasks.\n5. **Agentic reinforcement learning (Agentic RL):** train the model inside agent harnesses, such as[OpenClaw](https://openclaw.ai) ,[Hermes Agent](https://hermes-agent.nousresearch.com) ,[Pi](https://pi.dev) , and[OpenCode](https://opencode.ai) .\n\n## \n\t\tBenchmark results\n\t\n\nWe evaluated LFM2.5-24B-A2B against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, and Nemotron-3.5-Lightning on 16 benchmarks covering tool use and agentic workflows, knowledge, instruction following, math, and coding. It is the smallest model in the group, yet it is competitive with MoEs with ~50% more total and active parameters.\n\n| Benchmark | LFM2.5-24B-A2B | Qwen3.6-35B-A3B | Gemma-4-26B-A4B-it | Nemotron-3.5-Lightning (30B-A3B) | \n|---|---|---|---|---|\n| AutomationBench | **27.69** | 8.72 | 5.46 | 2.61 | \n| BFCLv4 | **65.01** | 65.00 | 57.09 | 53.78 | \n| ToolSandbox | **80.71** | 75.02 | 72.09 | 75.13 | \n| LiveClaw | **48.09** | 38.19 | 41.25 | 37.72 | \n| PinchBench | 83.18 | **89.93** | 77.12 | 84.20 | \n| τ³-Bench Banking | **8.04** | 7.63 | 4.54 | 7.01 | \n| WebArena Verified | 26.75 | **37.62** | 25.62 | 28.69 | \n| GPQA Diamond | 74.42 | **83.69** | 74.90 | 72.42 | \n| AA-Omniscience | -24.90 | -24.35 | -59.39 | **-3.58** | \n| HLE | 7.40 | 7.69 | **8.57** | 5.65 | \n| IFEval | **96.75** | 93.06 | 92.00 | 94.19 | \n| IFBench | **74.69** | 64.21 | 47.44 | 71.67 | \n| Multi-IF | **84.38** | 81.53 | 82.19 | 82.34 | \n| MATH-500 | 96.04 | **96.73** | 95.66 | 96.40 | \n| SWE-Bench Verified (OpenCode) | 51.00 | **66.20** | 25.80 | 53.40 | \n| SciCode | 32.71 | **37.35** | 35.42 | 27.78 | \n\n*All the models were evaluated with vLLM, using each benchmark's recommended generation parameters.\n\nFor your app, the strengths are tool use and instruction following. LFM2.5-24B-A2B leads or matches the competition on five of the eight tool-use and agentic benchmarks (AutomationBench, BFCLv4, ToolSandbox, LiveClaw, and τ³-Bench Banking), and on AutomationBench it scores more than three times the next-best model. It also leads on all three instruction-following benchmarks. Larger models keep an edge on knowledge-heavy benchmarks like GPQA Diamond, HLE, and AA-Omniscience, and on coding benchmarks like SWE-Bench Verified (OpenCode) and SciCode, so pair LFM2.5-24B-A2B with retrieval or web search for knowledge-heavy workloads.\n\nThese results make LFM2.5-24B-A2B a strong foundation for agents that need to reason, call tools, and run reliably on your device.\n\n## \n\t\tInference speed on CPU and GPU\n\t\n\nLFM2.5-24B-A2B ships with day-one support across the inference ecosystem, including llama.cpp, MLX, vLLM, SGLang, and ONNX.\n\n**Edge inference.** LFM2.5-24B-A2B is the fastest of the compared MoEs, decoding 184 tokens/s on an M5 Max and 114 tokens/s on a Ryzen AI Max+ 395. That is up to 1.7x faster than the next-best model, while using the least memory.\n\n**GPU inference.** LFM2.5-24B-A2B is the fastest model in its size class, reaching over 21K output tokens per second at high concurrency on a single B300.\n\n## \n\t\tHow to use LFM2.5-24B-A2B\n\t\n\nReach for LFM2.5-24B-A2B when you need agents that reason, call tools, and run reliably on your device.\n\nInstall the latest version of `transformers` (compatible with `transformers>=5.0.0`):\n\n```\npip install -U transformers\n```\n\nThen load and run the model:\n\n``` python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer\n\nmodel_id = \"LiquidAI/LFM2.5-24B-A2B\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    device_map=\"auto\",\n    dtype=\"bfloat16\",\n#   attn_implementation=\"flash_attention_2\" <- uncomment on compatible GPU\n)\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nstreamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)\n\nprompt = \"What is C. elegans?\"\n\ninput_ids = tokenizer.apply_chat_template(\n    [{\"role\": \"user\", \"content\": prompt}],\n    add_generation_prompt=True,\n    return_tensors=\"pt\",\n    tokenize=True,\n)[\"input_ids\"].to(model.device)\n\noutput = model.generate(\n    input_ids,\n    do_sample=True,\n    temperature=0.1,\n    top_k=50,\n    repetition_penalty=1.05,\n    max_new_tokens=512,\n    streamer=streamer,\n)\n```\n\n### \n\t\tUse it in your agent harness\n\t\n\nServe LFM2.5-24B-A2B locally behind any OpenAI-compatible backend, then point your agent harness at it. For example, with Hermes Agent:\n\n```\nhermes config set model.provider custom\nhermes config set model.base_url http://localhost:8080/v1\nhermes config set model.default LFM2.5-24B-A2B\nhermes config set model.context_length 131072\nhermes config set model.api_mode chat_completions\nhermes config set agent.tool_use_enforcement true\n```\n\nThe port depends on your serving backend: llama.cpp and MLX use `8080`, vLLM uses `8000`, SGLang uses `30000`, and LM Studio uses `1234`. See the [model card](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B) for OpenClaw and Pi configs, and [our guide](https://docs.liquid.ai/examples/agent-harnesses) for the full setup. Check out our [release blog](https://www.liquid.ai/blog/lfm2-5-24b-a2b) for video examples.\n\n## \n\t\tGet Started with LFM2.5-24B-A2B\n\t\n\nBoth LFM2.5-24B-A2B and LFM2.5-24B-A2B-Base are open-weight and available on Hugging Face today:\n\n- **Download:**[LFM2.5-24B-A2B-Base](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-Base) and[LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B) on Hugging Face.\n- **Use in your harness:** follow our[guide](https://docs.liquid.ai/examples/agent-harnesses) on how to run a local agent, like OpenClaw, Hermes Agent, and Pi.\n- **Fine-tune:** adapt LFM2.5-24B-A2B to your task with our[fine-tuning tutorials](https://github.com/Liquid4All/cookbook/tree/main/finetuning/notebooks) .\n\nWe can't wait to see what you build.\n\n## \n\t\tCitation\n\t\n\nIf you use this work, please cite the release blog:\n\n```\n@article{liquidAI202624BA2B,\n  author  = {Liquid AI},\n  title   = {LFM2.5-24B-A2B: Built for Fast On-Device Agents},\n  journal = {Liquid AI Blog},\n  year    = {2026},\n  note    = {www.liquid.ai/blog/lfm2-5-24b-a2b},\n}\n```\n\n", "url": "https://wpnews.pro/news/run-fast-private-agents-on-device-with-lfm2-5-24b-a2b", "canonical_source": "https://huggingface.co/blog/LiquidAI/lfm25-24b-a2b", "published_at": "2026-10-13 17:05:48+00:00", "updated_at": "2026-10-14 23:46:01.703069+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-research", "ai-infrastructure", "ai-tools"], "entities": ["LiquidAI", "LFM2.5-24B-A2B", "Qwen3.6-35B-A3B", "Gemma-4-26B-A4B-it", "Nemotron-3.5-Lightning", "OpenClaw", "Hermes Agent", "OpenCode"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/run-fast-private-agents-on-device-with-lfm2-5-24b-a2b", "markdown": "https://wpnews.pro/news/run-fast-private-agents-on-device-with-lfm2-5-24b-a2b.md", "text": "https://wpnews.pro/news/run-fast-private-agents-on-device-with-lfm2-5-24b-a2b.txt", "jsonld": "https://wpnews.pro/news/run-fast-private-agents-on-device-with-lfm2-5-24b-a2b.jsonld"}}