How I installed Radiance on my VM with one R9700 without the docker container A user documented installing the Radiance local LLM server from source on Fedora Server 44 with a single AMD R9700 GPU, skipping Docker, to run a 6-bit quantized Qwen 3.8 27B model. The build uses ROCm HIP with GPU target gfx1201, a 30000 MiB VRAM budget, an i8 KV cache, a 32768-token context length and 8 concurrent sequences, and the guide notes the MXFP4 container fails on this build, so an IQR-V4 container (iq2r–iq8r) is required instead. The write-up matters to builders running models on bare-metal AMD hardware who need a working source-build and configuration path outside containers. Look, I know Docker is an easy “push to start” for most things but for my environment where I have access to VMs and I’m running a hypervisor as-is rather than a docker-based server, it’s nice to actually get the bare metal setup. For this I am using Fedora Server 44 , and this comes with a few of its own quirks when building. I basically spent the day because I really wanted that “double performance” for running a Radiance specific model and Quen 3.8 27B is a pretty decent model. While it’s not “amazing” I am curious now how far I can push it with RAG tools and all kinds of other things. Recommended: Configure a static IP for your server This guide assumes the following hardware configuration: The reason is that the MXFP4 container fails on this build with either KV cache type: fp8 → i8 → The IQR-V4 family iq2r … iq8r is the int8-recipe line that the error itself points to. A single sentence — “Use an IQR-V4 container iq2r–iq8r , not the MXFP4 one; the MXFP4 export’s GDN weights are not served by this build” Without further ado, here we go. Install git, and development packages to build from source. sudo dnf install gcc gcc-c++ make cmake ninja-build git rocm-hip-devel rocm-smi hipcc sudo dnf group install -y c-development We want to do the following steps with this next shell segment /opt/radiance folder to contain the application cd /opt cloning makes the radiance folder git clone https://codeberg.org/StillDeadcode/radiance.git cd /opt/radiance Special fedora flag is DROCM PATH as /usr Most of this is default options, but to make absolute certain cmake -S . -B build -G Ninja \ -DCMAKE INSTALL PREFIX=/opt/radiance \ -DCMAKE BUILD WITH INSTALL RPATH=ON \ -DRAD WITH HIP=ON \ -DRAD GPU TARGETS='gfx1201' \ -DROCM PATH=/usr cmake --build build -j "$ nproc " sudo cmake --install build We need to poke a hole in the system firewall to allow the new service to communicate on port 8080. sudo firewall-cmd --permanent --new-service=radiance sudo firewall-cmd --permanent --service=radiance --add-port=8080/tcp sudo firewall-cmd --permanent --service=radiance --set-description="Radiance Local LLM Server" sudo firewall-cmd --reload Next we download the model we want in my example here, it’s the IPQ6 for a 6-bit quant . I made an entry in the /srv directory sudo mkdir -p /srv/models/qwen3.8-27b-iq6r cd /srv/models/qwen3.8-27b-iq6r Download from huggingface radiance adjacent repo curl -L -O https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/resolve/main/qwen3.8-27b-iq6r-v4.rad?download=true And now we can build the configuration yaml if you would like to test before we go ahead and make it a service. sudo vi /opt/radiance.yaml Here’s the file contents I used: radiance.yaml server: host: 192.168.x.x This is the listening IP, keep it local port: 8080 Optional: point to your build tree's plugins if you are running the binary directly from the build directory. radiance-home: /opt/radiance/build/radiance home memory: Set a VRAM budget slightly below your card's 32 GB total. The example from the provider uses 30000 MiB for a 32 GB card. gpu-memory-mib: 30000 models: qwen3.8-27b-iq6r: Path to your downloaded .rad container, or the Hugging Face repo ID. model: /srv/models/qwen3.8-27b-iq6r/qwen3.8-27b-iq6r-v4.rad Your single R9700 means tensor parallelism is 1. tp: 1 Use an 8-bit KV cache to save VRAM on a single card. kv-cache-dtype: i8 Starting context length; you can try increasing this later. max-model-len: 32768 Conservative number of concurrent sequences for a single GPU. max-num-seqs: 8 Optional but recommended: keep the token embedding on the card. embedding-placement: vram If you like, we can go ahead and run some checks against running the radiance service and hopefully catch any errors. We check the yaml then confirm it can start up with everything. /opt/radiance/bin/radiance --config /opt/radiance.yaml --check /opt/radiance/bin/radiance --config /opt/radiance.yaml As long as it starts without any errors, we are good to continue. Now we can configure the service. Create and edit the file /etc/systemd/system/radiance.service Unit Description=Radiance Local LLM Server After=network-online.target Wants=network-online.target Service Type=simple ExecStartPre=/opt/radiance/bin/radiance --config /opt/radiance.yaml --check ExecStart=/opt/radiance/bin/radiance --config /opt/radiance.yaml Restart=on-failure RestartSec=10 TimeoutStopSec=60 Raise file descriptor limit for many concurrent connections LimitNOFILE=65536 Locked memory for GPU pinned buffers optional, see note below LimitMEMLOCK=infinity Logging: everything goes to the journal StandardOutput=journal StandardError=journal Install WantedBy=multi-user.target Enable the service and turn it on. sudo systemctl daemon-reload sudo systemctl enable --now radiance.service sudo systemctl start radiance.service Now connect your favorite OpenAI API to the service and make sure it can read the models. Optionally, you can curl that my local server IP is 192.168.x.x . It’s important you don’t curl localhost because the server is only listening on the LAN Address of 192.168.x.x curl -s http://192.168.x.x:8080/v1/models