AMD Local LLM Setup on Windows/Linux: ROCm Overrides & Vulkan vs. HIP Benchmarking

A developer released two open-source Python tools aimed at simplifying local LLM inference on AMD GPUs. ROCmFix automatically resolves HSA_OVERRIDE_GFX_VERSION issues by querying PCI IDs via the Windows Registry or Linux lspci and setting environment variables across CMD, PowerShell, Bash, Zsh, and Fish, while InferBench benchmarks tokens-per-second and time-to-first-token across Vulkan and ROCm/HIP backends with forced VRAM unloads between runs.

The Problem with Local AI on AMD Hardware Running models locally with tools like Ollama or LM Studio on AMD GPUs often leads to two roadblocks: Missing drivers or unrecognised GPU architectures requiring HSA OVERRIDE GFX VERSION manual tweaks. Uncertainty over whether to run Vulkan backends or ROCm/HIP backends for maximum tokens per second. Solution 1: ROCmFix ROCmFix is a single-file Python utility designed to resolve HSA OVERRIDE GFX VERSION issues automatically. Key Features: Query PCI IDs directly using Windows Registry or Linux lspci. Set permanent or session environment variables for CMD, PowerShell, Bash, Zsh, and Fish. Run rocmfix doctor to inspect installed HIP SDK and Vulkan components. Quick run python rocmfix.py Solution 2: InferBench InferBench automates testing local LLM inference speeds across backend engines. Benchmarking Workflow: Executes warm-up queries. Forces VRAM unloads between runs to eliminate memory fragmentation/caching bias. Calculates median tok/s and TTFT Time-to-First-Token . Check out the repositories on GitHub: