# How I installed Radiance on my VM with one R9700 without the docker container

> Source: <https://forum.level1techs.com/t/how-i-installed-radiance-on-my-vm-with-one-r9700-without-the-docker-container/260912#post_1>
> Published: 2026-11-20 21:18:15+00:00

**Recommendation:** Build Radiance from source with -DRAD_WITH_HIP=ON and -DRAD_GPU_TARGETS='gfx1201', then serve an IQR-V4 quant with kv-cache-dtype i8 and a 30000 MiB VRAM budget on a single 32 GB card.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

Look, I know Docker is an easy “push to start” for most things but for my environment where I have access to VMs and I’m running a hypervisor as-is rather than a docker-based server, it’s nice to actually get the bare metal setup.

For this I am using **Fedora Server 44**, and this comes with a few of its own quirks when building.

I basically spent the day because I really wanted that “double performance” for running a Radiance specific model and Quen 3.8 27B is a pretty decent model. While it’s not “amazing” I am curious now how far I can push it with RAG tools and all kinds of other things.

**Recommended: Configure a static IP for your server**

**This guide assumes the following hardware configuration:**

The reason is that the MXFP4 container fails on this build with either KV cache type:

`fp8` → `i8` → The IQR-V4 family (`iq2r` … `iq8r` ) is the int8-recipe line that the error itself points to. A single sentence — *“Use an IQR-V4 container (iq2r–iq8r), not the MXFP4 one; the MXFP4 export’s GDN weights are not served by this build”*

Without further ado, here we go.

Install git, and development packages to build from source.

```
sudo dnf install gcc gcc-c++ make cmake ninja-build git rocm-hip-devel rocm-smi hipcc
sudo dnf group install -y c-development
```

We want to do the following steps with this next shell segment

`/opt/radiance` folder to contain the application

```
cd /opt
# cloning makes the radiance folder
git clone https://codeberg.org/StillDeadcode/radiance.git
cd /opt/radiance
# Special fedora flag is DROCM_PATH as /usr
# Most of this is default options, but to make absolute certain
cmake -S . -B build -G Ninja \
  -DCMAKE_INSTALL_PREFIX=/opt/radiance \
  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
  -DRAD_WITH_HIP=ON \
  -DRAD_GPU_TARGETS='gfx1201' \
  -DROCM_PATH=/usr
cmake --build build -j "$(nproc)"
sudo cmake --install build
```

We need to poke a hole in the system firewall to allow the new service to communicate on port 8080.

```
sudo firewall-cmd --permanent --new-service=radiance
sudo firewall-cmd --permanent --service=radiance --add-port=8080/tcp
sudo firewall-cmd --permanent --service=radiance --set-description="Radiance Local LLM Server"
sudo firewall-cmd --reload
```

Next we download the model we want (in my example here, it’s the IPQ6 for a 6-bit quant). I made an entry in the `/srv` directory

```
sudo mkdir -p /srv/models/qwen3.8-27b-iq6r
cd /srv/models/qwen3.8-27b-iq6r
# Download from huggingface (radiance adjacent repo)
curl -L -O https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/resolve/main/qwen3.8-27b-iq6r-v4.rad?download=true
```

And now we can build the configuration yaml if you would like to test before we go ahead and make it a service.

```
sudo vi /opt/radiance.yaml
```

Here’s the file contents I used:

```
# radiance.yaml
server:
  host: 192.168.x.x # This is the listening IP, keep it local
  port: 8080
  # Optional: point to your build tree's plugins if you are running
  # the binary directly from the build directory.
  # radiance-home: /opt/radiance/build/radiance_home

memory:
  # Set a VRAM budget slightly below your card's 32 GB total.
  # The example from the provider uses 30000 MiB for a 32 GB card.
  gpu-memory-mib: 30000

models:
  qwen3.8-27b-iq6r:
    # Path to your downloaded .rad container, or the Hugging Face repo ID.
    model: /srv/models/qwen3.8-27b-iq6r/qwen3.8-27b-iq6r-v4.rad
    # Your single R9700 means tensor parallelism is 1.
    tp: 1
    # Use an 8-bit KV cache to save VRAM on a single card.
    kv-cache-dtype: i8
    # Starting context length; you can try increasing this later.
    max-model-len: 32768
    # Conservative number of concurrent sequences for a single GPU.
    max-num-seqs: 8
    # Optional but recommended: keep the token embedding on the card.
    # embedding-placement: vram
```

If you like, we can go ahead and run some checks against running the radiance service and hopefully catch any errors.

We check the yaml then confirm it can start up with everything.

```
/opt/radiance/bin/radiance --config /opt/radiance.yaml --check
/opt/radiance/bin/radiance --config /opt/radiance.yaml
```

As long as it starts without any errors, we are good to continue.

Now we can configure the service.

Create and edit the file `/etc/systemd/system/radiance.service`

```
[Unit]
Description=Radiance Local LLM Server
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/opt/radiance/bin/radiance --config /opt/radiance.yaml --check
ExecStart=/opt/radiance/bin/radiance --config /opt/radiance.yaml
Restart=on-failure
RestartSec=10
TimeoutStopSec=60

# Raise file descriptor limit for many concurrent connections
LimitNOFILE=65536

# Locked memory for GPU pinned buffers (optional, see note below)
LimitMEMLOCK=infinity

# Logging: everything goes to the journal
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

Enable the service and turn it on.

```
sudo systemctl daemon-reload
sudo systemctl enable --now radiance.service
sudo systemctl start radiance.service
```

Now connect your favorite OpenAI API to the service and make sure it can read the models.

Optionally, you can curl that (my local server IP is 192.168.x.x). It’s important you don’t curl `localhost` because the server is only listening on the **LAN Address** of 192.168.x.x

```
curl -s http://192.168.x.x:8080/v1/models
```


