How I Got Ollama Running on My AMD Radeon RX 9060 XT Using WSL and ROCm

A developer detailed how to run AI models locally on an AMD Radeon RX 9060 XT using Windows Subsystem for Linux (WSL) and ROCm. The setup involves installing Ollama with its ROCm backend, configuring it as a systemd service, and exposing the GPU via librocdxg. The guide highlights version-specific issues and provides step-by-step commands for a successful local AI environment.

I wanted to start experimenting with AI for a while. At work, I didn't really have enough time to properly explore this new ecosystem and understand how these tools could be used. But I had another resource available: my personal hardware. I'm a gamer, so I have a reasonably powerful desktop PC. I'm also a developer, with a laptop running Ubuntu and a NAS. That made me wonder: Can I use my gaming PC to run AI models locally? The answer is yes . This article is a small walkthrough of my experience setting up a local AI environment using an AMD Radeon GPU , Windows , WSL , ROCm , and Ollama . My hardware is yes, I'm Team Red : The first step is to make sure that WSL is installed and up to date. The general process is: From PowerShell: Check the current version of WSL wsl -v Update WSL wsl --update Install Ubuntu wsl --install -d Ubuntu Start Ubuntu wsl -d Ubuntu Stop Ubuntu wsl --terminate Ubuntu Remove the Ubuntu distribution wsl --unregister Ubuntu Once Ubuntu is installed, update the system and install zstd . We will need it later to extract Ollama. sudo apt update sudo apt upgrade sudo apt autoclean sudo apt autoremove sudo apt install zstd Ollama is the foundation of this setup. It allows us to download and run AI models locally while providing different compute backends, including CPU, CUDA, and ROCm. One thing that wasn't immediately obvious to me was that Ollama provides a specific ROCm backend. The installation is split into two parts: Install Ollama curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst \ | sudo tar --zstd -x -C /usr Install the ROCm backend curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst \ | sudo tar --zstd -x -C /usr Then create the Ollama user and add your current user to the Ollama group: Create the Ollama user and group sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama Add the current user to the Ollama group sudo usermod -a -G ollama $ whoami The official documentation I used: https://docs.ollama.com/linux manual-install https://docs.ollama.com/linux manual-install https://docs.ollama.com/linux amd-gpu-install https://docs.ollama.com/linux amd-gpu-install I chose to run Ollama as a systemd service. Create or edit the service configuration: sudo nano /etc/systemd/system/ollama.service Unit Description=Ollama Service After=network-online.target Service ExecStart=/usr/local/bin/ollama serve User=ollama Group=ollama Restart=always RestartSec=3 Environment="PATH=$PATH" Required to detect the GPU through DXG Environment="HSA ENABLE DXG DETECTION=1" Optional: expose Ollama outside of WSL Environment="OLLAMA HOST=0.0.0.0:11434" Install WantedBy=default.target Then reload systemd and start the service: sudo systemctl daemon-reload sudo systemctl enable ollama sudo systemctl start ollama.service sudo systemctl status ollama.service More information about running Ollama as a service: https://docs.ollama.com/linux adding-ollama-as-a-startup-service-recommended https://docs.ollama.com/linux adding-ollama-as-a-startup-service-recommended The GPU needs to be exposed to the WSL environment. For my setup, I used librocdxg . One important detail: version 1.2.2 solved an issue I encountered with version 1.2.0 . So, if you're having issues with GPU detection, the version may matter. wget https://github.com/ROCm/librocdxg/releases/download/v1.2.2/rocdxg-roct 1.2.2 amd64.deb sudo apt install ./rocdxg-roct 1.2.2 amd64.deb More information about the project: https://github.com/ROCm/librocdxg https://github.com/ROCm/librocdxg At this point, everything should be configured. Restart Ollama: sudo systemctl restart ollama.service Then check the Ollama logs: sudo journalctl -u ollama --no-pager -o cat | grep "inference compute" You should see something similar to: library=ROCm compute=gfxXXXX description=" NAME OF YOUR GPU " The important part is: library=ROCm You don't want Ollama to silently fall back to CPU inference. In my case, I get: time=2026-08-28T12:38:20.900+02:00 level=INFO source=types.go:32 msg="inference compute" id=0 filter id=0 library=ROCm compute=gfx1200 name=ROCm0 description="AMD Radeon RX 9060 XT" libdirs=ollama,rocm v7 2 driver=0.0 pci id=0000:2b:00.0 type=discrete total="15.9 GiB" available="14.2 GiB" As you can see, Ollama detects: gfx1200 .AMD provides a compatibility matrix to identify which gfx version corresponds to your GPU: I also found references suggesting that it may be possible to enable support for some older gfx versions, but I didn't need to do this for my setup. Time for the interesting part. Download and run a model: ollama run qwen3:8b Ask the model something and let it generate a response. You can exit the interactive session with: /bye You can then check where the model is running: ollama ps The output should indicate that the model is running on the GPU. NAME ID SIZE PROCESSOR qwen3:8b xxxxxxxxxxxx 5.2 GB 100% GPU You can also open the Windows Task Manager and monitor your GPU usage. Once the model starts running, you should see: By default, your Ollama instance is running inside WSL. If you want to access it from other machines on your local network, you need to expose the port through Windows. On the Windows host, open PowerShell as Administrator. First, create a firewall rule: New-NetFirewallRule -DisplayName "Ollama WSL LAN" -Direction Inbound -Protocol TCP -LocalPort 11434 -Action Allow -RemoteAddress IP BASE TO LISTEN /24 Then forward the port between Windows and WSL: netsh interface portproxy add v4tov4 listenaddress= IP HOST WSL listenport=11434 connectaddress= IP UBUNTU IMAGE connectport=11434 From another computer on your local network, you can test the Ollama API with: curl.exe http:// IP OF OLLAMA HOST :11434/api/tags If everything is configured correctly, Ollama should respond with the list of installed models. By default, WSL may stop after some time when no active terminal is connected. To keep the WSL environment alive, create a .wslconfig file in your Windows user profile directory. Add: general instanceIdleTimeout=-1 After modifying the configuration, restart the WSL distribution so the configuration is taken into account Now once you started the image you will need to kill it manually. wsl --terminate Ubuntu I encountered one additional issue. After restarting my PC, I could start the WSL distribution, but Ollama was no longer accessible from the local network. In my case, restarting the Windows iphlpsvc service solved the problem. Open PowerShell as Administrator: Restart-Service iphlpsvc -Force After that, Ollama became accessible again. This setup allowed me to turn my gaming PC into a local AI machine without replacing Windows or dedicating the entire computer to Linux. The combination of: provides a relatively convenient environment for experimenting with local AI models. The most important part of the setup, in my experience, was making sure that Ollama was actually using the ROCm backend and the GPU. Once that was working, running a model such as qwen3:8b was straightforward. Hopefully, this will save some time for other AMD GPU users who want to experiment with local AI on Windows.