# Small LFM Models and Quantization

> Source: <https://www.dotnetperls.com/2026_9_10_small-lfm-models-quantization>
> Published: 2026-09-10 07:00:00+00:00

I have a local **MCP server** that mostly just calls a handful of functions. I had managed to get it working with a **350 million** parameter model (`LFM2.5-350M-QAD-Q4_0.gguf`), but could not go any lower. However I spent some time today to improve the code and now the 230M model (`LFM2.5-230M-Q5_K_M.gguf`) works.

The main thing I needed to do was change from **4-bit** quantization to **5-bit quantization**. It is a known issue that small models are more sensitive to quantization; and the 230M model seemed like it couldn't handle basic syntax at 4-bits.

The 5-bit version of the 230M model is **faster and smaller** than the 4-bit version of the 350M model. I can even get 500-1000 tokens per second on it, which is nearly instant.

For simple **tool-calling agents** the **LiquidAI 2.5 LFM models** are a good choice. These are models that can be run on low-end hardware (or really any hardware). And don't give up on the smallest model—the 230M model seems to be useful but apparently needs 5 bits per weight.