Ternary models represent each weight as one of 3 values (1, 0, and -1). They can be much smaller and faster than other quantizations but this comes at a price of quality. I downloaded the Bonsai 2 model (which is 7.2 GB) and used it in Pi code.
First, a special version of llama-cpp is currently needed and can be downloaded from PrismML's GitHub. I have a 12GB Nvidia GPU so the 7.2 GB model fits nicely; the 4-bit quantization of Qwen 3.8 would not fit. Bonsai 2 is based on Qwen 3.8 27B, a popular local coding model.
I found:
• Bonsai 2 27B works as an agentic coder, and it rarely fails tool calls, although it tends to over-think like the original Qwen 3.8 model does.
• Bonsai 2 27B is a significant improvement over the previous Bonsai 27B models in quality of output.
• The claim that Bonsai 2 27B is only 2% degraded from Qwen 3.8 27B may be at least somewhat accurate in general.
• On an RTX 3060 desktop card with 12 GB RAM the model decodes at around 30 tokens per second, which is usable (although the fan is loud).
• Bonsai 2 is useful for real coding tasks and may help alleviate the need to buy GPUs with huge amounts of memory.