Ternary Bonsai 2 Usage Tips

A user improved the agentic coding performance of Ternary Bonsai 2, a Qwen 3.8 27B-based model, when run locally through llama-cpp by switching its thinking setting from "xhigh" to medium via the chat-template-kwargs argument and enabling self-speculation with the spec-default argument. The user reported that the medium thinking setting curbed excessively long reasoning-token generation, while speculative decoding based on previously generated tokens helped because the model repeatedly outputs similar code blocks while thinking. The user called Ternary Bonsai 2 probably the most powerful model they have run locally, with the fewest quantization-related errors.

I spent some time using llama-cpp with Ternary Bonsai 2 . Over the last couple days I had determined that this model, based on Qwen 3.8 27B, has potential for agentic coding, so I decided to try to improve its performance a bit. I changed the thinking to medium from " xhigh " as it tended to generate reasoning tokens way too long. This can be done with the chat-template-kwargs argument in llama-cpp . Next, I added self-speculation with the spec-default argument to llama-cpp . This enables speculative decoding based on tokens previously generated. For coding, as in Pi Code, this really helps Qwen and Ternary Bonsai 2 as it repeatedly outputs similar code blocks while thinking. With these 2 changes, I improved performance of agentic coding and it was somewhat less frustrating . Ternary Bonsai 2 is probably the most powerful model I have been able to run locally, with the fewest quantization related errors.