{"slug": "why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels", "title": "Why Ollama runs on CPU after an update: the missing sm_86 CUDA kernels", "summary": "A developer discovered that Ollama 0.32.14 silently falls back to CPU inference on NVIDIA GPUs with compute capability 8.6 (such as RTX 30-series and A6000) because the bundled CUDA kernels omit sm_86 and the CUDA 12 fallback path broke. The issue is tracked in GitHub issue #17841, and the developer recommends pinning to version 0.32.13 until a fix is released.", "body_md": "You update Ollama, pull the model you use every day, and everything gets slower. Not broken, not erroring. Just slow. `ollama ps`\n\nsays something like `12% CPU / 88% GPU`\n\n, so you shrug and carry on. Then you open nvidia-smi mid-generation and see Ollama using 0 MiB of VRAM.\n\nIt's been running on CPU this whole time. Nobody told you.\n\nThis is what's happening to a lot of RTX 30-series owners after Ollama 0.32.14, and to people with A40/A6000/A10 cards too.\n\nThe giveaway is buried in the server log. Somewhere in the noise:\n\n```\nmsg=\"skipping CUDA device - compute capability not in compiled architectures\"\ndevice=\"NVIDIA RTX A6000\" cc=860 archs=\"[750 890 1000 1200]\"\n```\n\ncc=860 means your GPU is compute capability 8.6. The arch list is the set of architectures the bundled CUDA kernels were compiled for: 7.5 (RTX 20-series), 8.9 (RTX 40-series), 10.0 and 12.0. No 8.6. Your card isn't in the build.\n\nAffected hardware: every sm_86 card. RTX 3090, 3080, 3070, 3060, plus the A40, A6000, A5000, A10 and friends. A big chunk of the people running local LLMs are on exactly these cards.\n\nThe missing sm_86 kernel isn't new. Older builds skipped the same arch. But they had a safety net: when the CUDA 13 kernels didn't cover the card, they fell back to the bundled CUDA 12 library, which does include sm_86. Same skip line, then:\n\n```\nmsg=\"inference compute\" ... library=CUDA compute=8.6\n```\n\nIn 0.32.14 that fallback path broke. So instead of dropping to the CUDA 12 lib, the runner skips straight to `library=cpu`\n\n. Nothing errors, so you don't notice until the token rate starts to hurt.\n\nUpstream hasn't shipped a fix as of writing. The issue is open and the only maintainer reply so far is a request for more logs. The dependable move: pin to the last version where the CUDA 12 fallback worked, which for this one is 0.32.13.\n\nWindows:\n\n`ollama`\n\nservice if you run it as one.Linux:\n\n```\nsudo systemctl stop ollama\n# install the v0.32.13 .deb / .rpm / tarball from the releases page\nsudo systemctl start ollama\n```\n\nThis is the step everyone skips, and honestly the reason this post exists. `ollama ps`\n\ncan lie. It showed a GPU split while the GPU sat idle. The reliable check:\n\n`nvidia-smi`\n\nduring a generation. VRAM allocated to the ollama process, nonzero, means real GPU.`library=CUDA compute=8.6`\n\nagain, not `library=cpu`\n\n.`CUDA_VISIBLE_DEVICES=0`\n\n. The GPU is visible. It's being skipped at the arch check, which is compile-time. No env var brings a missing arch back.`OLLAMA_LOAD_LIBRARY`\n\n. Some builds respect it, but in 0.32.14 the fallback path itself was the broken part, so don't count on it for this version.After any update to a local-model stack (Ollama, llama.cpp, LM Studio, whatever), spend 30 seconds proving acceleration before you trust it. One `nvidia-smi`\n\nduring a generation is the cheapest smoke test there is.\n\nSilent degradation is worse than an error. An error tells you something's wrong. A silent CPU fallback just makes everything sluggish and you end up blaming the model.\n\nAnd prebuilt-kernel coverage is decided at build time. When a release drops your GPU architecture, there's no flag that fixes it. Pin the last good version, watch the release notes, and move back up when the coverage returns.\n\nThe thread that started this: [https://github.com/ollama/ollama/issues/17841](https://github.com/ollama/ollama/issues/17841)", "url": "https://wpnews.pro/news/why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels", "canonical_source": "https://dev.to/milkyway008/why-ollama-runs-on-cpu-after-an-update-the-missing-sm86-cuda-kernels-39bn", "published_at": "2026-08-22 17:05:59+00:00", "updated_at": "2026-08-22 17:43:34.399088+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "artificial-intelligence"], "entities": ["Ollama", "NVIDIA", "RTX 3090", "RTX 3080", "RTX 3070", "RTX 3060", "A6000", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels", "markdown": "https://wpnews.pro/news/why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels.md", "text": "https://wpnews.pro/news/why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels.txt", "jsonld": "https://wpnews.pro/news/why-ollama-runs-on-cpu-after-an-update-the-missing-sm-86-cuda-kernels.jsonld"}}