I spent some time using llama-cpp with Ternary Bonsai 2. Over the last couple days I had determined that this model, based on Qwen 3.8 27B, has potential for agentic coding, so I decided to try to improve its performance a bit.
I changed the thinking to medium from "xhigh" as it tended to generate reasoning tokens way too long. This can be done with the chat-template-kwargs argument in llama-cpp.
Next, I added self-speculation (with the spec-default argument) to llama-cpp. This enables speculative decoding based on tokens previously generated. For coding, as in Pi Code, this really helps Qwen (and Ternary Bonsai 2) as it repeatedly outputs similar code blocks while thinking.
With these 2 changes, I improved performance of agentic coding and it was somewhat less frustrating. Ternary Bonsai 2 is probably the most powerful model I have been able to run locally, with the fewest quantization related errors.