{"slug": "how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container", "title": "How I installed Radiance on my VM with one R9700 without the docker container", "summary": "A user documented installing the Radiance local LLM server from source on Fedora Server 44 with a single AMD R9700 GPU, skipping Docker, to run a 6-bit quantized Qwen 3.8 27B model. The build uses ROCm HIP with GPU target gfx1201, a 30000 MiB VRAM budget, an i8 KV cache, a 32768-token context length and 8 concurrent sequences, and the guide notes the MXFP4 container fails on this build, so an IQR-V4 container (iq2r–iq8r) is required instead. The write-up matters to builders running models on bare-metal AMD hardware who need a working source-build and configuration path outside containers.", "body_md": "Look, I know Docker is an easy “push to start” for most things but for my environment where I have access to VMs and I’m running a hypervisor as-is rather than a docker-based server, it’s nice to actually get the bare metal setup.\n\nFor this I am using **Fedora Server 44**, and this comes with a few of its own quirks when building.\n\nI basically spent the day because I really wanted that “double performance” for running a Radiance specific model and Quen 3.8 27B is a pretty decent model. While it’s not “amazing” I am curious now how far I can push it with RAG tools and all kinds of other things.\n\n**Recommended: Configure a static IP for your server**\n\n**This guide assumes the following hardware configuration:**\n\nThe reason is that the MXFP4 container fails on this build with either KV cache type:\n\n`fp8` → `i8` → The IQR-V4 family (`iq2r` … `iq8r` ) is the int8-recipe line that the error itself points to. A single sentence — *“Use an IQR-V4 container (iq2r–iq8r), not the MXFP4 one; the MXFP4 export’s GDN weights are not served by this build”*\n\nWithout further ado, here we go.\n\nInstall git, and development packages to build from source.\n\n```\nsudo dnf install gcc gcc-c++ make cmake ninja-build git rocm-hip-devel rocm-smi hipcc\nsudo dnf group install -y c-development\n```\n\nWe want to do the following steps with this next shell segment\n\n`/opt/radiance` folder to contain the application\n\n```\ncd /opt\n# cloning makes the radiance folder\ngit clone https://codeberg.org/StillDeadcode/radiance.git\ncd /opt/radiance\n# Special fedora flag is DROCM_PATH as /usr\n# Most of this is default options, but to make absolute certain\ncmake -S . -B build -G Ninja \\\n  -DCMAKE_INSTALL_PREFIX=/opt/radiance \\\n  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \\\n  -DRAD_WITH_HIP=ON \\\n  -DRAD_GPU_TARGETS='gfx1201' \\\n  -DROCM_PATH=/usr\ncmake --build build -j \"$(nproc)\"\nsudo cmake --install build\n```\n\nWe need to poke a hole in the system firewall to allow the new service to communicate on port 8080.\n\n```\nsudo firewall-cmd --permanent --new-service=radiance\nsudo firewall-cmd --permanent --service=radiance --add-port=8080/tcp\nsudo firewall-cmd --permanent --service=radiance --set-description=\"Radiance Local LLM Server\"\nsudo firewall-cmd --reload\n```\n\nNext we download the model we want (in my example here, it’s the IPQ6 for a 6-bit quant). I made an entry in the `/srv` directory\n\n```\nsudo mkdir -p /srv/models/qwen3.8-27b-iq6r\ncd /srv/models/qwen3.8-27b-iq6r\n# Download from huggingface (radiance adjacent repo)\ncurl -L -O https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/resolve/main/qwen3.8-27b-iq6r-v4.rad?download=true\n```\n\nAnd now we can build the configuration yaml if you would like to test before we go ahead and make it a service.\n\n```\nsudo vi /opt/radiance.yaml\n```\n\nHere’s the file contents I used:\n\n```\n# radiance.yaml\nserver:\n  host: 192.168.x.x # This is the listening IP, keep it local\n  port: 8080\n  # Optional: point to your build tree's plugins if you are running\n  # the binary directly from the build directory.\n  # radiance-home: /opt/radiance/build/radiance_home\n\nmemory:\n  # Set a VRAM budget slightly below your card's 32 GB total.\n  # The example from the provider uses 30000 MiB for a 32 GB card.\n  gpu-memory-mib: 30000\n\nmodels:\n  qwen3.8-27b-iq6r:\n    # Path to your downloaded .rad container, or the Hugging Face repo ID.\n    model: /srv/models/qwen3.8-27b-iq6r/qwen3.8-27b-iq6r-v4.rad\n    # Your single R9700 means tensor parallelism is 1.\n    tp: 1\n    # Use an 8-bit KV cache to save VRAM on a single card.\n    kv-cache-dtype: i8\n    # Starting context length; you can try increasing this later.\n    max-model-len: 32768\n    # Conservative number of concurrent sequences for a single GPU.\n    max-num-seqs: 8\n    # Optional but recommended: keep the token embedding on the card.\n    # embedding-placement: vram\n```\n\nIf you like, we can go ahead and run some checks against running the radiance service and hopefully catch any errors.\n\nWe check the yaml then confirm it can start up with everything.\n\n```\n/opt/radiance/bin/radiance --config /opt/radiance.yaml --check\n/opt/radiance/bin/radiance --config /opt/radiance.yaml\n```\n\nAs long as it starts without any errors, we are good to continue.\n\nNow we can configure the service.\n\nCreate and edit the file `/etc/systemd/system/radiance.service`\n\n```\n[Unit]\nDescription=Radiance Local LLM Server\nAfter=network-online.target\nWants=network-online.target\n\n[Service]\nType=simple\nExecStartPre=/opt/radiance/bin/radiance --config /opt/radiance.yaml --check\nExecStart=/opt/radiance/bin/radiance --config /opt/radiance.yaml\nRestart=on-failure\nRestartSec=10\nTimeoutStopSec=60\n\n# Raise file descriptor limit for many concurrent connections\nLimitNOFILE=65536\n\n# Locked memory for GPU pinned buffers (optional, see note below)\nLimitMEMLOCK=infinity\n\n# Logging: everything goes to the journal\nStandardOutput=journal\nStandardError=journal\n\n[Install]\nWantedBy=multi-user.target\n```\n\nEnable the service and turn it on.\n\n```\nsudo systemctl daemon-reload\nsudo systemctl enable --now radiance.service\nsudo systemctl start radiance.service\n```\n\nNow connect your favorite OpenAI API to the service and make sure it can read the models.\n\nOptionally, you can curl that (my local server IP is 192.168.x.x). It’s important you don’t curl `localhost` because the server is only listening on the **LAN Address** of 192.168.x.x\n\n```\ncurl -s http://192.168.x.x:8080/v1/models\n```\n\n", "url": "https://wpnews.pro/news/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container", "canonical_source": "https://forum.level1techs.com/t/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container/260912#post_1", "published_at": "2026-11-20 21:18:15+00:00", "updated_at": "2026-11-20 21:42:21.501540+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "mlops"], "entities": ["Radiance", "Fedora Server 44", "AMD R9700", "Qwen 3.8 27B", "ROCm", "Hugging Face", "Docker"], "also_reported_by": [], "takeaway": "On a single R9700 running Radiance built from source with ROCm target gfx1201, use an IQR-V4 container (iq2r–iq8r) rather than MXFP4, since the MXFP4 export's GDN weights are not served by this build, and set gpu-memory-mib to 30000 with an i8 KV cache.", "recommendation": {"advice": "Build Radiance from source with -DRAD_WITH_HIP=ON and -DRAD_GPU_TARGETS='gfx1201', then serve an IQR-V4 quant with kv-cache-dtype i8 and a 30000 MiB VRAM budget on a single 32 GB card.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container", "markdown": "https://wpnews.pro/news/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container.md", "text": "https://wpnews.pro/news/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container.txt", "jsonld": "https://wpnews.pro/news/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container.jsonld"}}