Welcome everyone.
Just got curious of the NPU discussion and can chime in.
my strix halo can run image generation much faster and with very low impact to the performance using an image model on the npu, the quality is not there when it comes to text, but all other things considered i like the results along side the qwen 3.8 flash next on halogen:
| # | Model, engine | Time per image | Memory | Text in image | LLM while generating (1 / 4 sessions) |
|---|---|---|---|---|---|
| 1 | Z-Image-Turbo Q8, stable-diffusion.cpp Vulkan (GPU) | 31 s | ~10.7 GB | perfect | collapses (64→10 / 75→9 tok/s) |
| 2 | Z-Image-Turbo Q4_K, sd.cpp Vulkan (GPU) | 31 s | ~8 GB | perfect | collapses (same) |
| 3 | FLUX.2-klein-4B Q8, sd.cpp Vulkan (GPU) | 13 s | ~7.9 GB | garbled | (GPU: same problem) |
| 4 | FLUX.2-klein-4B, Halogen on the NPU (production) | 9.6 s | 7.6 GB | nearly correct | 55 / 64 tok/s (−10–15 %) |
So people can generate images to basically no cost of inference speed. I like it.
source & further reading
forum.level1techs.com — original article
SpawnDev.ILGPU.ML: Neural network inference & training in C# across any GPU (Browser & Desktop)
Virtualization workstation with the Intel Arc Pro B50
Quadro RTX 6000 suddenly overheating, even after heatsink repaste