Bonsai 1.7B for MCP Server

A hands-on comparison of two local LLMs for driving a custom MCP server found that Bonsai 1.7B, at 248.3 MB, was more reliable at tool calling than the smaller LFM 2.5 350M at 219.3 MB, despite running at roughly 250 tokens per second versus over 500 tokens per second for the 350M model. Both models fit within a 300 MB budget and called tools correctly, but the author reported the 350M model occasionally produced incorrect tool arguments. The author concluded that more parameters quantized down is the better trade-off for reliability, while lower-parameter models remain preferable where speed matters most.

Now that I have a custom MCP server running locally, I need an LLM running in llama-cpp to control it. I have found most any model will work; the main challenge is finding the fastest, smallest, and most accurate one. This brings us to the age-old question of choosing a local LLM: is quantization level or parameter count more important? If I allocate a budget of 300 MB for the model, I can use either a model like LFM 2.5 350M which is 219.3 MB or Bonsai 1.7B which is 248.3 MB . Both models work and can call tools correctly. So which is better? I guess this question comes down to testing and preferences. The LFM model is faster; I can get over 500 tokens per second. Meanwhile Bonsai 1.7B is closer to 250 tokens per second. I feel like Bonsai 1.7B tends to be more reliable —occasionally the 350M model can make some sort of mistake that slows me down like an incorrect tool argument or something . So at least for myself, I feel like models with more parameters that are quantized down are a better option, due to increased reliability; but other use cases may favor the lower-parameter models they are faster .