cd /news/large-language-models/how-i-installed-radiance-on-my-vm-wi… · home › topics › large-language-models › article
[ARTICLE · art-173487] src=forum.level1techs.com ↗ pub= topic=large-language-models verified=true sentiment=· neutral

How I installed Radiance on my VM with one R9700 without the docker container

A user documented installing the Radiance local LLM server from source on Fedora Server 44 with a single AMD R9700 GPU, skipping Docker, to run a 6-bit quantized Qwen 3.8 27B model. The build uses ROCm HIP with GPU target gfx1201, a 30000 MiB VRAM budget, an i8 KV cache, a 32768-token context length and 8 concurrent sequences, and the guide notes the MXFP4 container fails on this build, so an IQR-V4 container (iq2r–iq8r) is required instead. The write-up matters to builders running models on bare-metal AMD hardware who need a working source-build and configuration path outside containers.

recommendation

Build Radiance from source with -DRAD_WITH_HIP=ON and -DRAD_GPU_TARGETS='gfx1201', then serve an IQR-V4 quant with kv-cache-dtype i8 and a 30000 MiB VRAM budget on a single 32 GB card.

more in playbook: Inference and self-hosting
read4 min views2 publishedNov 20, 2026

Look, I know Docker is an easy “push to start” for most things but for my environment where I have access to VMs and I’m running a hypervisor as-is rather than a docker-based server, it’s nice to actually get the bare metal setup.

For this I am using Fedora Server 44, and this comes with a few of its own quirks when building.

I basically spent the day because I really wanted that “double performance” for running a Radiance specific model and Quen 3.8 27B is a pretty decent model. While it’s not “amazing” I am curious now how far I can push it with RAG tools and all kinds of other things.

Recommended: Configure a static IP for your server

This guide assumes the following hardware configuration:

The reason is that the MXFP4 container fails on this build with either KV cache type:

fp8 → i8 → The IQR-V4 family (iq2r … iq8r ) is the int8-recipe line that the error itself points to. A single sentence — “Use an IQR-V4 container (iq2r–iq8r), not the MXFP4 one; the MXFP4 export’s GDN weights are not served by this build”

Without further ado, here we go.

Install git, and development packages to build from source.

sudo dnf install gcc gcc-c++ make cmake ninja-build git rocm-hip-devel rocm-smi hipcc
sudo dnf group install -y c-development

We want to do the following steps with this next shell segment

/opt/radiance folder to contain the application

cd /opt
git clone https://codeberg.org/StillDeadcode/radiance.git
cd /opt/radiance
cmake -S . -B build -G Ninja \
  -DCMAKE_INSTALL_PREFIX=/opt/radiance \
  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
  -DRAD_WITH_HIP=ON \
  -DRAD_GPU_TARGETS='gfx1201' \
  -DROCM_PATH=/usr
cmake --build build -j "$(nproc)"
sudo cmake --install build

We need to poke a hole in the system firewall to allow the new service to communicate on port 8080.

sudo firewall-cmd --permanent --new-service=radiance
sudo firewall-cmd --permanent --service=radiance --add-port=8080/tcp
sudo firewall-cmd --permanent --service=radiance --set-description="Radiance Local LLM Server"
sudo firewall-cmd --reload

Next we download the model we want (in my example here, it’s the IPQ6 for a 6-bit quant). I made an entry in the /srv directory

sudo mkdir -p /srv/models/qwen3.8-27b-iq6r
cd /srv/models/qwen3.8-27b-iq6r
curl -L -O https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/resolve/main/qwen3.8-27b-iq6r-v4.rad?download=true

And now we can build the configuration yaml if you would like to test before we go ahead and make it a service.

sudo vi /opt/radiance.yaml

Here’s the file contents I used:

server:
  host: 192.168.x.x # This is the listening IP, keep it local
  port: 8080

memory:
  gpu-memory-mib: 30000

models:
  qwen3.8-27b-iq6r:
    model: /srv/models/qwen3.8-27b-iq6r/qwen3.8-27b-iq6r-v4.rad
    tp: 1
    kv-cache-dtype: i8
    max-model-len: 32768
    max-num-seqs: 8

If you like, we can go ahead and run some checks against running the radiance service and hopefully catch any errors.

We check the yaml then confirm it can start up with everything.

/opt/radiance/bin/radiance --config /opt/radiance.yaml --check
/opt/radiance/bin/radiance --config /opt/radiance.yaml

As long as it starts without any errors, we are good to continue.

Now we can configure the service.

Create and edit the file /etc/systemd/system/radiance.service

[Unit]
Description=Radiance Local LLM Server
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/opt/radiance/bin/radiance --config /opt/radiance.yaml --check
ExecStart=/opt/radiance/bin/radiance --config /opt/radiance.yaml
Restart=on-failure
RestartSec=10
TimeoutStopSec=60

LimitNOFILE=65536

LimitMEMLOCK=infinity

StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable the service and turn it on.

sudo systemctl daemon-reload
sudo systemctl enable --now radiance.service
sudo systemctl start radiance.service

Now connect your favorite OpenAI API to the service and make sure it can read the models.

Optionally, you can curl that (my local server IP is 192.168.x.x). It’s important you don’t curl localhost because the server is only listening on the LAN Address of 192.168.x.x

curl -s http://192.168.x.x:8080/v1/models
── more in #large-language-models 4 stories · sorted by recency
── more on @radiance 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-i-installed-radi…] indexed:0 read:4min 2026-11-20 · —