This is my experience after one week of running local AI on Strix Halo, in mid-August, 2026.
I’ll put the TL; DR up front: The hardware is fine, but the software stack is far from there yet.
I have 2 setups under my control, one at work and one at home, with a bit different hardware.
This one is sitting there doing nothing right now as I am trying to come up with things to run on there
Business ordering Framework turned out to be a complete pain and it took about a month to get all the crap sorted and for us to receive the machines. However since this is about strix halo I’m not going to elaborate here.
Knowing that support for Deepseek V4 on vLLM is nonexistent on AMD, I started with llama.cpp. However with llama.cpp and ROCm backend I was hitting a bug where it crashes at about 4000 tokens input when using rpc (basically useless). Vulkan crashed at about 6000 ish, still basically useless.
I then switched to ds4 (purpose built engine for deepseek v4), which doesn’t crash but since it doesn’t support batching on AMD or true pipeline parallelism it means one node is always sitting there doing nothing at any given moment.
Take these with a grain of salt, they weren’t properly benchmarked
llama.cpp rocm:
Prefill I forgot, Decode about 12.5t/s single stream, 16t/s aggregate (unsloth Q8 + BF16 Dspark)
though this one is basically useless since it crashes at ~4000 tokens context
llama.cpp Vulkan:
Prefill I forgot, Decode about 8t/s single stream, 11t/s aggregate (unsloth Q8 + BF16 Dspark)
ds4:
Prefill 60tok/s, Decode 12.5t/s (the ds4 specific weights they provide)
Because we weren’t completely sure if we wanted this or not when we ordered these. AMD gave us the best exit strategy as these are very useful as general purpose workstations, unlike the main other options (Apple M5 Max and Nvidia GB10) which are basically useless for us outside AI inference). If I were to pick something to run local LLMs on today I definitely would not pick AMD for sure.
I’m not blaming one but here are the stupid things I had to deal with along the way
It genuinely feels like AMD is being treated as a 3rd class citizen on all the major AI stuff, after Nvidia and Apple.
Nice hardware being held back severely by software.
I don’t know what I should run on the home setup currently, maybe throw some claude tokens at some of the problems I faced since I have some free tokens. Or maybe you guys can suggest things to do.