Reliability and Small Models

Models with fewer than 1 billion parameters, such as LFM 2.5 350M, exhibit unreliable tool calling in agentic systems, prompting developers to implement repair mechanisms and prompt-checking systems in MCP servers. The approach involves double-processing the user prompt to catch details the small model might miss, improving reliability at the cost of extra logic.

Large models with 2 billion or more parameters such as Gemma 4 E4B tend to be reliable in tool calling in agentic systems. But as one tries smaller and smaller models, reliability drops. Sub-1B models can behave in unexpected ways; some additional logic is needed when using them. In an MCP server, it is often necessary to repair or correct tool calls done by these smaller models . This became more necessary as I started tested a 350B parameter model LFM 2.5 350M . I also started using a prompt-checking system: • The agent reports the last user prompt back as a parameter to the MCP server. • The MCP server uses code to check things in the original prompt validating or finding information that the model might have missed . So in essence we have a double-processing system. The model is supposed to process the user prompt , but then the MCP server does it again just to improve reliability and find things the model might have missed. It is sort of like cheating—but the smallest models are less reliable, and cannot be trusted to find all important details from an input.