Today I downloaded and used Qwen 3.5 0.8B (Qwen3.5-0.8B-Q4_K_M.gguf) on my local system. I wanted to see if it would work on my MCP server, which I had designed for small models (it did). I will add it to my collection of useful models.
In using it as an agentic function-calling model, I found that a temperature of 0 is helpful. The documentation suggests higher temperatures like 0.7 or 1.0, but 0 seems to be most consistent and reliable. So if you try out 0.8B, make sure to lower the temperature before deleting it.
• It thinks by default, which is interesting for a 752M parameter model.
• Using MTP on Qwen 0.8B seems to be either slower or not noticeably faster—probably no point in trying to speed up a 532 MB model.
• It is bigger and slower than the LFM 2.5 models I usually run, but it seems like it may be more capable (this makes sense, as it has three times more parameters).
• It is nice to have another model that can handle my workload; Qwen 3.5 0.8B is a valued addition to my model collection.