{"slug": "small-language-models-350m-parameters", "title": "Small Language Models (350M Parameters)", "summary": "LiquidAI's 350M-parameter small language model required extensive code rewrites but achieved 500 tokens per second, compared to 300 tokens per second for the 1.2B model, according to a developer's tests with MCP servers. The 230M model proved too small for the use case, while the 1.2B and 2.6B models worked with minimal modifications.", "body_md": "I have had good results with **small language models** (such as 1.2B parameters) for interacting with my MCP server. I decided to try to see how small I could go—if 1.2B worked, would 350M work, would 230M? Each size decrease means **more efficiency**.\n\nI have found LiquidAI's small models to be quite **effective** at calling tools. I could use the **1.2B** parameter model with minimal changes to the MCP server code. With **350M**, I had to **redesign** a feature and use a different approach, but it eventually worked as well.\n\nSome notes:\n\n• While 2.6B (`LFM2.5-2.6B-QAD-Q4_0.gguf`\n\n) and 1.2B (`LFM2.5-1.2B-Instruct-QAD-Q4_0.gguf`\n\n) work well with minimal modifications, 350M (`LFM2.5-350M-QAD-Q4_0.gguf`\n\n) required fairly extensive code changes and rewrites.\n\n• 230M (`LFM2.5-230M-QAD-Q4_0.gguf`\n\n) does not appear viable for my use—it is simply too small.\n\n• The performance improvements with small models are astonishing—a 1.2B model can do 300 tokens a second, while a 350M model can do 500 tokens a second.\n\nHow can one build a **program** that **works well** with a 350M model? Basically I recommend designing it from **scratch** to work with the model—test each feature, and only add features that the model can handle. A 350M parameter model has limitations; it is possible to build up a program that works with it though.", "url": "https://wpnews.pro/news/small-language-models-350m-parameters", "canonical_source": "https://www.dotnetperls.com/2026_8_22_small-language-models-350m", "published_at": "2026-08-22 07:00:00+00:00", "updated_at": "2026-08-22 14:13:02.605129+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence"], "entities": ["LiquidAI", "LFM2.5-350M-QAD-Q4_0.gguf", "LFM2.5-1.2B-Instruct-QAD-Q4_0.gguf", "LFM2.5-2.6B-QAD-Q4_0.gguf", "LFM2.5-230M-QAD-Q4_0.gguf"], "alternates": {"html": "https://wpnews.pro/news/small-language-models-350m-parameters", "markdown": "https://wpnews.pro/news/small-language-models-350m-parameters.md", "text": "https://wpnews.pro/news/small-language-models-350m-parameters.txt", "jsonld": "https://wpnews.pro/news/small-language-models-350m-parameters.jsonld"}}