cd /news/artificial-intelligence/unsloth-desktop-npu-support-brings-3… · home › topics › artificial-intelligence › article
[ARTICLE · art-172799] src=hwbusters.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

Unsloth Desktop NPU Support Brings 35B Local Models to AMD Ryzen AI Laptops on Windows and Linux

AMD and Unsloth announced Unsloth Desktop NPU support for AMD's XDNA 2 engine, letting Ryzen AI laptops and mini PCs run local LLMs on the NPU under Windows 11 and x64 Linux and expose them to more than 100 agent apps through an OpenAI-compatible endpoint. Unsloth's published figures show Qwen3.6-35B-A3B at 20.6 GB, 17.2 tokens/s and 3.13 s to first token, while the dense Qwen3.8-27B runs at 2.0 tokens/s, indicating memory bandwidth rather than NPU TOPS is the limit. Models must be converted to the 4-bit Q4NX format for FastFlowLM, and Unsloth claims NPUs need two to four times less energy per generated token than a GPU, a vendor figure the tables do not attribute to a specific chip or power mode.

key takeaway

On Ryzen AI hardware, prefer MoE models like Qwen3.6-35B-A3B (about 3B active parameters, 17.2 tokens/s) over dense ones like Qwen3.8-27B (2.0 tokens/s), and budget RAM accordingly (64 GB recommended for the 35B).

by read3 min views6 publishedNov 19, 2026
Unsloth Desktop NPU Support Brings 35B Local Models to AMD Ryzen AI Laptops on Windows and Linux
Image: Hwbusters (auto-discovered)

Unsloth Desktop can now run local LLMs on the XDNA 2 NPU inside Ryzen AI laptops and mini PCs, on Windows 11 and Linux, and expose them to more than 100 agent apps through an OpenAI-style API.

AMD and Unsloth announced today that Unsloth Desktop NPU support has arrived for AMD’s XDNA 2 engine, making it the first NPU family the app can drive. Until now, the neural block in a Ryzen AI chip has mostly been a spec-sheet number, useful for Windows Studio Effects and not much else. This is one of the clearest attempts yet to give it a real job.

The idea is simple. Unsloth Desktop runs a local, OpenAI-compatible endpoint. An app sends it a request, Lemonade and FastFlowLM handle the model on the NPU, and the answer goes back to the app. AMD names Hermes Agent, OpenClaw, Open WebUI, Cline, LangChain and GitHub Copilot among the compatible tools, and counts more than 100 apps and workflows in total. The pitch is sustained, low-power inference that leaves the CPU and the Radeon iGPU free for other work.

What it runs on, and how fast #

According to Unsloth’s setup guide, the supported chips are Ryzen AI 300 (Strix Point and Krackan Point), Ryzen AI Max (Strix Halo) and Ryzen AI 400 (Gorgon Point). Windows 11 works, and so does x64 Linux, which is the more interesting half. There you need kernel 7.0 or newer with the in-tree amdxdna driver, or the DKMS package on an older kernel, plus current NPU firmware and an unlimited memlock limit. Unsloth checks all of that when you click “Enable NPU” in the model picker and tells you what is missing.

Models use Q4NX, a 4-bit format built for FastFlowLM. Unsloth publishes its own figures for the current lineup:

  • Qwen3.6-35B-A3B : 20.6 GB, 64 GB of RAM recommended, 17.2 tokens/s, 3.13 s to first token
  • Qwen3.8-27B : 18.1 GB, 48 GB recommended, 2.0 tokens/s
- **Gemma4-12B-IT** : 8.7 GB, 32 GB recommended, 6.8 tokens/s
- **Qwen3.5-9B** : 7.6 GB, 32 GB recommended, 9.5 tokens/s
- **LFM3-1.7B** : 1.5 GB, 16 GB recommended, 40.7 tokens/s

The spread tells the story. The 35B mixture-of-experts model only activates about 3B parameters per token, so it runs at a usable 17 tokens/s. The dense 27B model crawls at 2 tokens/s. Memory bandwidth is the wall here, not the NPU’s TOPS rating, and anyone shopping for a Ryzen AI box to run agents locally should plan around MoE models and as much RAM as they can afford.

The fine print #

Unsloth claims NPUs need two to four times less energy per generated token than a GPU. That is plausible for this class of silicon, but it is the vendor’s claim, and the tables don’t say which chip or power mode produced the throughput numbers. FastFlowLM ships with four power modes, Performance being the default, so expect lower figures on battery.

You also can’t load an arbitrary GGUF onto the NPU yet. It has to be converted to Q4NX first, either on Unsloth’s Export page, where the conversion runs on the CPU, or with AMD’s open-source converter on GitHub. On the security side, Unsloth pins and SHA-256 checks its runtime downloads, and its managed Lemonade server only accepts local connections with a fresh auth key on every start. If you’d rather keep the NPU out of it entirely, setting UNSLOTH_STUDIO_ENABLE_NPU=0 before launch turns the feature off.

FastFlowLM itself was folded into ROCm back in August, so the inference engine isn’t new. What’s new is the packaging. A Strix Halo system can now serve a 35B model from the NPU while the GPU fine-tunes or renders in the background, and that is exactly the split AMD has been promising since it first put an NPU in a Ryzen.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @amd 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/unsloth-desktop-npu-…] indexed:0 read:3min 2026-11-19 · —