{"slug": "ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl", "title": "Ollama Not Using GPU? Fix It on Linux, Windows and WSL", "summary": "A developer published a troubleshooting guide explaining how to diagnose why Ollama falls back to CPU inference instead of using a GPU on Linux, Windows, and WSL. The guide centers on the `ollama ps` PROCESSOR column and the server log's \"inference compute\" line, then walks through five common causes: outdated NVIDIA drivers, missing `render` and `video` group membership for AMD ROCm on Linux, pinned `OLLAMA_LLM_LIBRARY` environment variables, containers started without GPU access, and VRAM shortfalls that force partial offload.", "body_md": "Run `ollama ps` while a model is loaded: the PROCESSOR column tells you the truth. `100% GPU` means the GPU is fine and you can stop reading. A split like `40%/60% CPU/GPU` means the model didn't fit in VRAM — use a smaller quant. `100% CPU` means Ollama found no usable GPU: usually an outdated driver, a missing group membership (AMD on Linux), a pinned `OLLAMA_LLM_LIBRARY`, or a container started without GPU access. Fix the cause the logs name; there are only about five.\n\nTwo commands, no guessing.\n\n```\n# In one terminal: load a model\nollama run llama3.2 \"hello\"\n\n# In another: see where it runs\nollama ps\nNAME            ID          SIZE     PROCESSOR        UNTIL\nllama3.2:latest a80c4de17cd9 3.3 GB   100% GPU         4 minutes from now\n```\n\nThe PROCESSOR column has three states:\n\n`100% GPU` — every layer offloaded. Done.`48%/52% CPU/GPU` — partial offload. The GPU is working, but the model plus context didn't fit in VRAM. See the VRAM section below.`100% CPU` — inference is on the CPU. The GPU was either not detected or deliberately disabled.\nThen read the server log, which names the hardware Ollama actually found at startup:\n\n```\njournalctl -u ollama --no-pager | grep -i \"inference compute\"\n```\n\nOn a healthy NVIDIA box you want a line like:\n\n```\ninference compute id=GPU-xxxx library=CUDA compute=8.9 driver=12.4 name=NVIDIA GeForce RTX 4070\n```\n\nNo line at all, or one that ends with a CPU-only fallback message, and you've found your problem. The rest of this post is the five causes, most likely first.\n\nOn NVIDIA, the usual culprit is the driver, not CUDA. Ollama ships its own CUDA runtime libraries, so you do not need the CUDA toolkit installed — but the bundled runtime needs a driver new enough to talk to it. `nvidia-smi` working is not proof; it only proves a driver exists, not that it's recent enough.\n\n```\nnvidia-smi --query-gpu=driver_version --format=csv,noheader\n```\n\nIf the version is years old, update it and reboot:\n\n```\n# Debian/Ubuntu family\nsudo apt install nvidia-driver-570\n# Arch family\nsudo pacman -S nvidia\n```\n\nAfter a driver update, restart the Ollama service so it re-detects devices — detection happens once at startup, not per request:\n\n```\nsudo systemctl restart ollama\n```\n\nIf the log now prints your GPU with `library=CUDA`, you're done. If it still refuses, check that `OLLAMA_LLM_LIBRARY` isn't set anywhere — see the \"after an update\" section.\n\nPartial offload is arithmetic, not a bug: the model weights plus the KV cache for your context window must fit in VRAM. A 7B model at Q4 is roughly 4–5 GB; give it an 8K context and the cache adds more. On an 8 GB card something has to stay on the CPU, and `ollama ps` shows the split.\n\nThree ways to close the gap, cheapest first:\n\n`num_ctx` dominates the cache size. 32K context on an 8 GB card means most layers stay on CPU.`num_gpu` option caps how many layers get offloaded. Setting it below the layer count guarantees a split — if someone set it in a Modelfile or API call, unset it.\nNote the reverse trap too: a GPU that shows `100% GPU` but runs *slower* than expected may be swapping over system RAM. Check `ollama ps` SIZE against your actual VRAM.\n\nAMD on Linux needs three things, and all three are checkable:\n\n**1. ROCm support in the build.** The official Linux install script bundles a ROCm build. Confirm what the server detected:\n\n```\njournalctl -u ollama --no-pager | grep -iE \"rocm|inference compute\"\n```\n\n**2. Group membership.** The ROCm runtime needs access to `/dev/kfd` and `/dev/dri`, which means the `render` and `video` groups:\n\n```\nsudo usermod -aG render,video $USER\n# log out and back in, then:\nsudo systemctl restart ollama\n```\n\nThis single missing group is the most common \"Ollama not using GPU on Ubuntu\" post on every forum, and it survives driver reinstalls because the driver was never the problem.\n\n**3. A supported GPU — or an override.** Unsupported RDNA2 consumer cards (gfx1031, gfx1032) fail detection even with a working ROCm stack. The standard workaround is claiming a compatible target:\n\n```\nsudo systemctl edit ollama\n[Service]\nEnvironment=\"HSA_OVERRIDE_GFX_VERSION=10.3.0\"\n```\n\nThen `sudo systemctl restart ollama`. This is an unsupported-but-widely-used override; if it misbehaves, remove it and you're back to official support territory. If you'd rather have full control over backends than fight autodetection, that's the core difference covered in [llama.cpp vs Ollama](https://dev.to/en/blog/2026-09-10/llama-cpp-vs-ollama).\n\nOn Windows, AMD support is narrower — check Ollama's supported-GPU list for your card before assuming the install is broken.\n\nUpdates change one of three things, in this order of likelihood:\n\n`OLLAMA_LLM_LIBRARY` forces a specific runner (`cuda_v11`, `rocm`, or even `cpu`). It's meant for debugging, it overrides autodetection silently, and it persists in shell profiles and service files long after the reason is forgotten. Find it and remove it:\n\n```\nsystemctl show ollama --property=Environment | grep -i llm_library\nenv | grep OLLAMA\ndocker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 ollama/ollama\n```\n\nFor AMD containers, the equivalent is device passthrough plus group adds:\n\n```\ndocker run -d --device=/dev/kfd --device=/dev/dri \\\n  --group-add video --group-add render \\\n  -v ollama:/root/.ollama -p 11434:11434 ollama/ollama\n```\n\nNo `--gpus=all`, no GPU — Docker has no reason to be generous.\n\nYes, with the right driver in the right place: install the **Windows** NVIDIA driver, never a Linux driver inside the distro — an in-WSL driver breaks CUDA passthrough rather than fixing it. Then update WSL itself and confirm the passthrough device exists:\n\n```\nwsl --update   # from PowerShell\nls /dev/dxg    # inside WSL — must exist for GPU use\n```\n\nWith `/dev/dxg` present and a current Windows driver, Ollama in WSL2 offloads to the GPU like a native install. If you'd rather skip the indirection entirely, the Windows build of Ollama runs natively and sees the GPU without WSL.\n\nNo — a CPU-only run is functionally identical, just slower, and for small models on a fast CPU it can be perfectly usable. On Apple Silicon the question dissolves: Metal uses unified memory automatically, and the only limit is how much RAM you're willing to share with the model.\n\n| Symptom | Likely cause | Fix | \n|---|---|---|\n| `100% CPU` in`ollama ps` , NVIDIA card present | Driver too old for bundled CUDA | Update driver, reboot, restart service | \n| `100% CPU` , AMD on Linux | Missing `render` /`video` group | `usermod -aG render,video` , re-login | \n| `100% CPU` , unsupported AMD card | ROCm rejects the gfx target | `HSA_OVERRIDE_GFX_VERSION=10.3.0` | \n| `40%/60% CPU/GPU` split | Model + context exceed VRAM | Smaller quant or shorter `num_ctx` | \n| GPU worked yesterday, CPU today | Pinned `OLLAMA_LLM_LIBRARY` or stale driver | Find and remove the env var; update driver | \n| GPU in native runs, CPU in Docker | Container launched without GPU flags | Recreate with `--gpus=all` (or AMD devices) | \n\nCheck in this order: `ollama ps` for the state, server logs for the detection list, then the table. Nine times out of ten the log line already told you which row you're in.", "url": "https://wpnews.pro/news/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl", "canonical_source": "https://dev.to/mrsaynothing/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl-3jb8", "published_at": "2026-09-16 20:03:39+00:00", "updated_at": "2026-09-16 20:53:19.245859+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "large-language-models", "developer-tools"], "entities": ["Ollama", "NVIDIA", "CUDA", "AMD", "ROCm", "llama3.2", "Ubuntu", "WSL"], "alternates": {"html": "https://wpnews.pro/news/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl", "markdown": "https://wpnews.pro/news/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl.md", "text": "https://wpnews.pro/news/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl.txt", "jsonld": "https://wpnews.pro/news/ollama-not-using-gpu-fix-it-on-linux-windows-and-wsl.jsonld"}}