How to Run LFM3-1.7B Locally on a Phone or Laptop

LFM3-1.7B fits in roughly 1GB at 4-bit precision. Here's how to run it via GGUF, MLX, ONNX, or Ollama on phones and laptops.

What is LFM3-1.7B and why does it matter for local inference? #

LFM3-1.7B is a 1.73-billion-parameter text model from Liquid AI, built on what the company calls a tri-hybrid architecture mixing gated short convolutions with sliding-window and global attention layers. At 4-bit precision it takes up about 1GB of memory, small enough to run on a phone or a laptop without a discrete GPU. It ships with a 256K-token context window, support for 16 languages, and post-training aimed at agentic use cases like tool calling and structured output rather than open-ended chat or heavy reasoning.

TL;DR #

• Memory footprint drops to around 1GB at 4-bit quantization, which is why Liquid AI positions LFM3-1.7B as a model that fits on consumer phones and laptops rather than requiring a cloud GPU.
• Architecture mix combines 13 gated short convolution layers, 5 sliding-window attention layers, and 6 global attention layers across 24 total layers, a departure from pure transformer designs aimed at efficiency on constrained hardware.
• Multiple quantization formats are available out of the box: BF16 for full-precision GPU serving, FP8 and NVFP4 for optimized GPU inference, and GGUF/MLX/ONNX for CPU, Apple Silicon, and cross-platform local deployment.
• Agentic focus means the model is post-trained for harnesses like Hermes Agent, Pi, and OpenClaw, with native tool calling, deferred tool for large tool catalogs, and a 256K context window.
• Not built for everything : Liquid AI explicitly says it’s a poor fit for agentic coding or complex multi-step reasoning given its size, and recommends pairing it with retrieval for knowledge-heavy tasks.
• Fine-tuning is single-GPU friendly , with published notebooks covering supervised fine-tuning, DPO, GRPO, and continued pretraining for tasks like structured extraction or adding a new language.

How do you pick the right format for your hardware? #

The choice of format depends almost entirely on what hardware you’re targeting:

• GGUF : the go-to format for CPU and mixed CPU/GPU inference through llama.cpp, Ollama, or LM Studio. This is the most practical starting point if you want to run LFM3-1.7B on a laptop without worrying about driver or framework compatibility.
• MLX (4-bit) : built specifically for Apple Silicon. If you’re on an M-series Mac, this format takes advantage of the unified memory architecture and Apple’s MLX framework for faster local inference than a generic CPU path.
• ONNX : a cross-platform runtime format, useful when you need the model to run consistently across different operating systems or embed it in an application via ONNX Runtime rather than a chat-focused tool.
• FP8 and NVFP4 : these are GPU-only quantization formats intended for serving through SGLang, not for laptop or phone deployment. FP8 roughly halves memory versus BF16, while NVFP4 (a 4-bit floating-point format) pushes further for GPU inference where throughput matters more than squeezing onto edge hardware.
• BF16 : the full-precision checkpoint, meant for GPU serving through vLLM, SGLang, or Hugging Face Transformers when you have the VRAM to spare and want the highest-fidelity outputs.

For anyone specifically trying to run the model on a phone or a laptop without a dedicated GPU, GGUF (via Ollama or llama.cpp) and MLX (on Mac) are the two formats that matter.

What does it take to actually run LFM3-1.7B on a laptop? #

Because the model needs about 1GB of RAM at 4-bit precision, the hardware bar is low. Any modern laptop, including machines without a discrete GPU, has enough memory to load the model alongside a typical operating system and other applications. On Apple Silicon Macs, the MLX 4-bit build is the natural choice since it’s tuned for that chip family. On Windows or Linux machines, or older Macs, GGUF through Ollama or LM Studio is simpler to set up: both tools handle quantized GGUF files directly and provide a chat interface or local API without manual environment configuration.

For developers who want programmatic access rather than a chat UI, the Transformers library works directly with the model using a standard pipeline call, the model by name and running inference with trust_remote_code=True. Liquid AI’s documentation recommends specific sampling settings depending on the task: lower temperature (0.1) with top_k=50 and a repetition penalty of 1.1 for general chat, tool use, and coding, versus a higher temperature (1.0) with top_p=0.95 and top_k=20 for math and STEM tasks.

How does the tool-calling and agent setup work? #

LFM3-1.7B uses a ChatML-style chat template with structured tags for tool calls, formatted as XML inside <tool_call> blocks. Developers pass function schemas through the tools parameter when applying the chat template, and the model generates calls that the application then executes and feeds back as a tool message in the conversation history.

Remy is new. The platform isn't. #

Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.

A notable feature is deferred tool , aimed at agent harnesses with large tool catalogs. Instead of stuffing every tool definition into the prompt, tools can be marked with defer_=True so their definitions stay out of context until a search mechanism retrieves them by name. This matters for real-world agent deployments, like the GRPO fine-tuning example in Liquid AI’s own cookbook, which trains the model to pick the right function out of a 108-tool phone app. Keeping the full tool list out of every prompt reduces token overhead and keeps the context window usable for actual conversation and reasoning.

The model also supports a reasoning mode by default, surfaced through <think> tags in the chat template, which can be disabled with enable_thinking=False, though Liquid AI notes this toggle can behave inconsistently on harder reasoning tasks.

Is LFM3-1.7B worth using instead of a larger model? #

It depends on the job. At under 2 billion parameters, LFM3-1.7B is not going to compete with larger frontier models on complex reasoning or multi-step coding tasks, and Liquid AI says as much directly in its own documentation. Where it’s a reasonable choice is for local, latency-sensitive, or privacy-sensitive agentic tasks: tool calling, structured data extraction, domain-specific assistants, or applications where running inference entirely on-device (without a network round trip to a cloud API) is the actual requirement.

The model’s 256K context window is generous for its size, which helps with agent harnesses that accumulate a lot of tool definitions, conversation history, or retrieved documents. But Liquid AI recommends pairing it with retrieval rather than expecting the model to carry broad world knowledge, since a 1.7B-parameter model simply doesn’t have the capacity to memorize as much as larger models do.

Fine-tuning is where the model’s size becomes an advantage rather than a limitation. Liquid AI provides example notebooks for supervised fine-tuning (structured JSON extraction from messy documents), DPO (answering only from retrieved documents or declining), GRPO (tool selection in a large tool catalog), and continued pretraining (learning a new language from raw text). All of these are described as feasible on a single GPU, which is a meaningfully lower bar than fine-tuning larger models.

Frequently Asked Questions #

How much RAM does LFM3-1.7B need to run locally?

At 4-bit quantization, Liquid AI states the model takes about 1GB of memory, which is low enough to run on most modern phones and laptops alongside normal background applications.

What’s the difference between GGUF and MLX for running this model?

GGUF is a cross-platform format used by tools like llama.cpp, Ollama, and LM Studio, and works on CPUs across operating systems. MLX is a 4-bit format built specifically for Apple Silicon, taking advantage of the M-series chips’ unified memory architecture.

Can LFM3-1.7B handle coding tasks?

Liquid AI explicitly advises against using it for agentic coding or complex multi-step reasoning, citing its small parameter count. It’s better suited to tool calling, structured output, and domain-specific tasks after fine-tuning.

What context length does LFM3-1.7B support?

It supports a 256K-token context window covering both input and output, which is large relative to its 1.7B parameter size and useful for agent harnesses that need to hold onto tool definitions and conversation history.

Does LFM3-1.7B support multiple languages?

Yes. Liquid AI lists support for 16 languages including English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, and Polish.