“They” say necessity is the mother of invention…
Overall, I’ve been pleased with my purchase of a 128GB Strix Halo (Bosgame) but the prefill has been killing me. So when Qwen3.8 27B dropped, I blew the last week or so of my life in front of Claude trying to make it usable for agentic workflows.
I honestly had more success than expected.
I ended up building an inference engine tailored to the Strix Halo and Qwen 3.8 architecture… It turned out being about 2x as fast at prefill as the next closest engine I could find at higher model weights. >550t/s @32k depth. There might be a little more room to go but it is approaching the wall of physics and my hardware can’t run past 2400Mhz under load. If you have good cooling and can hit closer to the 2900MHz clock, you can probably push 600+ @32k context depth.
Anyway, I wasn’t going to release this, but decided if even a few people find it useful and it saves them time its worth it.
I tried to make it as simple as possible to just fire up a server and run it. The instructions are in the repo. Model weights on hugging face.
I’m using this with my own harness (in development) and it is definitely usable. Is it Deepseek v4 flash fast? No. But once you warm it up and you’re careful with context its been extremely useful and enjoyable to work with. The engine does prompt caching so it feels pretty good.
If you have a Strix Halo, try it out and let me know what you think. I hope it saves you time. Thanks for looking.
P.S - I don’t have social media or reddit so this is probably the only place I’m going to post this. If you want to leave feedback just put it in this thread, or as an issue on github.
repo is at github / peonist-ai/halogen-server