I have a local MCP server that mostly just calls a handful of functions. I had managed to get it working with a 350 million parameter model (LFM2.5-350M-QAD-Q4_0.gguf), but could not go any lower. However I spent some time today to improve the code and now the 230M model (LFM2.5-230M-Q5_K_M.gguf) works.
The main thing I needed to do was change from 4-bit quantization to 5-bit quantization. It is a known issue that small models are more sensitive to quantization; and the 230M model seemed like it couldn't handle basic syntax at 4-bits.
The 5-bit version of the 230M model is faster and smaller than the 4-bit version of the 350M model. I can even get 500-1000 tokens per second on it, which is nearly instant.
For simple tool-calling agents the LiquidAI 2.5 LFM models are a good choice. These are models that can be run on low-end hardware (or really any hardware). And don't give up on the smallest model—the 230M model seems to be useful but apparently needs 5 bits per weight.