Small Models for Agentic Systems

LiquidAI's LFM 2.5 1.2B, a 1.2-billion-parameter model that is 696 MB on disk, can power agentic systems with modifications to an MCP server, according to a developer's test. The model required adding examples and removing a confusing tool, but it runs at over 300 tokens per second and uses minimal RAM, potentially eliminating the need for GPU upgrades for local agentic tasks.

Last night I spent some time trying to make my MCP server work with a smaller local model , one that is just 1.2 billion parameters and 696 MB on disk. This is LiquidAI's LFM 2.5 1.2B, with file name LFM2.5-1.2B-Instruct-QAD-Q4 0.gguf . Most small models I test fail too badly and I just delete them right away. However this model seemed to have some potential to work for my agentic system. It required some changes in the MCP server. • I had to add more examples and details about how to respond on certain user queries. • I had to remove a confusing tool which it did not seem to understand, and would randomly call just to annoy me . So basically I have a system with just 1 tool that can be called in many different ways. The small model thus does not get confused about what it is supposed to do. It still gets confused by certain things: it has trouble keeping track of who said what, and what certain words mean in a phrase. However it runs at over 300 tokens per second, uses minimal RAM, and will make it so I basically will never need to update my GPU to run local agentic tasks it even will run speedily on CPU alone .