# AMD Local LLM Setup on Windows/Linux: ROCm Overrides & Vulkan vs. HIP Benchmarking

> Source: <https://dev.to/xanpavle/amd-local-llm-setup-on-windowslinux-rocm-overrides-vulkan-vs-hip-benchmarking-15p2>
> Published: 2026-09-19 21:47:27+00:00

The Problem with Local AI on AMD Hardware

Running models locally with tools like Ollama or LM Studio on AMD GPUs often leads to two roadblocks:

Missing drivers or unrecognised GPU architectures requiring HSA_OVERRIDE_GFX_VERSION manual tweaks.

Uncertainty over whether to run Vulkan backends or ROCm/HIP backends for maximum tokens per second.

**Solution 1: ROCmFix**

ROCmFix is a single-file Python utility designed to resolve `HSA_OVERRIDE_GFX_VERSION issues` automatically.

*Key Features:*

Query PCI IDs directly using Windows Registry or Linux lspci.

Set permanent or session environment variables for CMD, PowerShell, Bash, Zsh, and Fish.

Run rocmfix doctor to inspect installed HIP SDK and Vulkan components.

`# Quick run`

python rocmfix.py

**Solution 2: InferBench**

InferBench automates testing local LLM inference speeds across backend engines.

*Benchmarking Workflow:*

Executes warm-up queries.

Forces VRAM unloads between runs to eliminate memory fragmentation/caching bias.

Calculates median tok/s and TTFT (Time-to-First-Token).

Check out the repositories on GitHub: