This is my very first post, hello there.
So my Strix Halo is a 128GB Bosgame M5. I got it used for 1800€ at Ebay. I didn’t do any hardware changes to it yet, except exchanging the thermal paste with Arctic MX7, which brought my idle temps down a bit. Before the fan was audible every few minutes, now it’s silent in idle. Worth it.
I run Proxmox 9 and ROCm 7.13 from the AMD repository. A LLM LXC container has Debian 13 and Docker from the Docker repository installed. It runs another LXC with Hermes and a VM with some other stuff, 8GB RAM.
For extra speed i installed the ROCmFPX llama.cpp fork and use quantization to the ROCmFP4 data type, a must have if you use Strix Halo.
Here is my start command for Qwen 3.8 27b, i get 30 t/s decode:
/root/ROCmFPX/build-strix-rocmfp4/bin/llama-server \
--hf-repo kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF --model Qwen3.8-27B-Q4_0_ROCMFP4_STRIX.gguf --alias Qwen-3.8-27b \
--hf-repo-draft kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF --model-draft mtp-Qwen3.8-27B-Q4_0.gguf --spec-type draft-mtp --spec-draft-ngl 99 --spec-draft-device ROCm0 --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
-ngl 999 --jinja -fit off --parallel 2 -dev ROCm0 \
--ctx-size 131072 --flash-attn on --kv-unified --cache-type-k q8_0 --cache-type-v q8_0 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0 \
--cache-ram 4096 \
--reasoning-budget 16384 --reasoning-budget-message " ... thinking budget exceeded, let's answer now." \
--no-mmap --host 0.0.0.0 --port 1338 \
--metrics
Another model i run is Ling 3.0 Flash. I get 50 t/s decode with this config:
/root/ROCmFPX/build-strix-rocmfp4/bin/llama-server \
--hf-repo raulvidis/Ling-3.0-flash-ROCmFP4-STRIX-MTP-GGUF:Q4_0 --alias Ling-3.0-Flash \
--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0.5 \
--temp 0.6 --top-p 0.95 --top-k 20 --jinja \
--cache-ram 4096 \
-dev ROCm0 --webui \
--host 0.0.0.0 --port 1337 \
--gpu-layers 999 -fa on \
--ctx-size 196608 --flash-attn on --kv-unified --cache-type-k q8_0 --cache-type-v q8_0 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0 \
-fit off -np 4 --no-mmap
Qwen 3.8 27b is a dense model, and so it’s pretty slow. The config i have feels fast enough for me. Ling 3.0 Flash is where the fun begins. It is, in my opinion, far better than Qwen 3.6 35b and is underrated.
I know that many don’t like the Artificial Intelligence Index, but i use it as a guide to find models worth my while. Ling 3.0 Tiny, Ling 3.0 Flash and Deepseek V4 Flash 0713 are particularly good for Strix Halo as they have a pretty small amount of active parameters.