{"slug": "amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking", "title": "AMD Local LLM Setup on Windows/Linux: ROCm Overrides & Vulkan vs. HIP Benchmarking", "summary": "A developer released two open-source Python tools aimed at simplifying local LLM inference on AMD GPUs. ROCmFix automatically resolves HSA_OVERRIDE_GFX_VERSION issues by querying PCI IDs via the Windows Registry or Linux lspci and setting environment variables across CMD, PowerShell, Bash, Zsh, and Fish, while InferBench benchmarks tokens-per-second and time-to-first-token across Vulkan and ROCm/HIP backends with forced VRAM unloads between runs.", "body_md": "The Problem with Local AI on AMD Hardware\n\nRunning models locally with tools like Ollama or LM Studio on AMD GPUs often leads to two roadblocks:\n\nMissing drivers or unrecognised GPU architectures requiring HSA_OVERRIDE_GFX_VERSION manual tweaks.\n\nUncertainty over whether to run Vulkan backends or ROCm/HIP backends for maximum tokens per second.\n\n**Solution 1: ROCmFix**\n\nROCmFix is a single-file Python utility designed to resolve `HSA_OVERRIDE_GFX_VERSION issues` automatically.\n\n*Key Features:*\n\nQuery PCI IDs directly using Windows Registry or Linux lspci.\n\nSet permanent or session environment variables for CMD, PowerShell, Bash, Zsh, and Fish.\n\nRun rocmfix doctor to inspect installed HIP SDK and Vulkan components.\n\n`# Quick run`\n\npython rocmfix.py\n\n**Solution 2: InferBench**\n\nInferBench automates testing local LLM inference speeds across backend engines.\n\n*Benchmarking Workflow:*\n\nExecutes warm-up queries.\n\nForces VRAM unloads between runs to eliminate memory fragmentation/caching bias.\n\nCalculates median tok/s and TTFT (Time-to-First-Token).\n\nCheck out the repositories on GitHub:", "url": "https://wpnews.pro/news/amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking", "canonical_source": "https://dev.to/xanpavle/amd-local-llm-setup-on-windowslinux-rocm-overrides-vulkan-vs-hip-benchmarking-15p2", "published_at": "2026-09-19 21:47:27+00:00", "updated_at": "2026-09-19 21:54:31.595061+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "large-language-models", "developer-tools", "mlops"], "entities": ["AMD", "ROCm", "ROCmFix", "InferBench", "Ollama", "LM Studio", "HIP", "Vulkan"], "alternates": {"html": "https://wpnews.pro/news/amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking", "markdown": "https://wpnews.pro/news/amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking.md", "text": "https://wpnews.pro/news/amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking.txt", "jsonld": "https://wpnews.pro/news/amd-local-llm-setup-on-windows-linux-rocm-overrides-vulkan-vs-hip-benchmarking.jsonld"}}