# Deploying Custom YOLOv8 on Raspberry Pi 5 + Hailo-8L

> Source: <https://gist.github.com/mi0iou/87c4b5315e4e0edee0437c7e04290cdb>
> Published: 2026-04-12 18:20:10+00:00

# Compiling Custom YOLOv8 HEFs on Non-Ubuntu Linux Using the Hailo Docker (No Shell Script)

**A practical guide for compiling custom-trained YOLOv8 models into HEF files using the official Hailo AI Software Suite Docker image — without needing Ubuntu, a Hailo PCIe device, or nvidia-docker on your host.**

**Target:** Raspberry Pi 5 + Hailo-8L (13 TOPS) AI HAT+  
**Pipeline:** `.pt → .onnx → .har → .hef → deploy on Pi 5`  
**Tested with:** Hailo AI SW Suite 2025-10, Docker 28.x, Linux Mint (should work on any Linux with Docker)

---

## Why This Guide Exists

The official Hailo documentation assumes Ubuntu and provides a shell script (`hailo_ai_sw_suite_docker_run.sh`) that probes for PCIe drivers and nvidia-docker before launching the container. If you're compiling on a workstation that doesn't have a physical Hailo device installed — which is most people's setup — that script causes unnecessary problems. This guide bypasses it entirely with a direct `docker run` command.

**What you need on the host:**
- Docker (that's it)
- 16 GB+ RAM (32 GB recommended)
- 25 GB+ free disk space
- An NVIDIA GPU for training (not needed for compilation — the DFC runs on CPU)

**What you do NOT need on the host:**
- Ubuntu
- Hailo PCIe driver
- HailoRT
- nvidia-docker / NVIDIA Container Toolkit
- A physical Hailo device

---

## Phase 1: Train Your Model

Standard Ultralytics training — nothing Hailo-specific here:

```python
from ultralytics import YOLO

model = YOLO('yolov8n.pt')  # or yolov8s.pt
model.train(
    data='your_dataset.yaml',
    epochs=100,
    imgsz=640,
    batch=16,
    device=0
)
# Result: runs/detect/train/weights/best.pt
```

## Phase 2: Export to ONNX

```python
from ultralytics import YOLO

model = YOLO('runs/detect/train/weights/best.pt')
model.export(format='onnx', imgsz=640, opset=11, simplify=True)
# Result: runs/detect/train/weights/best.onnx
```

**Critical notes:**
- **`opset=11`** — Required by the Hailo DFC. Higher opset versions cause parsing failures.
- **`imgsz=640`** — Must match your training input size.
- **`simplify=True`** — Runs onnx-simplifier to remove redundant ops.
- **Do NOT add NMS** to the ONNX — the Hailo pipeline handles NMS on the host CPU during inference.

## Phase 3: Prepare Calibration Images

The DFC needs representative images from your dataset to calibrate INT8 quantisation.

```bash
mkdir -p ~/hailo_compile/models
mkdir -p ~/hailo_compile/calib

cp runs/detect/train/weights/best.onnx ~/hailo_compile/models/

# Copy a random subset of training images (just images, no labels):
ls /path/to/dataset/train/images/ | shuf -n 1000 | \
    xargs -I{} cp /path/to/dataset/train/images/{} ~/hailo_compile/calib/
```

- **256 minimum, 1000+ recommended.**
- Images do NOT need to be pre-resized — the compiler handles resizing.
- Should be representative of real-world deployment conditions.
- Just raw images — no labels or annotations needed.

**Why calibration matters:** The DFC converts your model from FP32 to INT8 (from ~4 billion possible values per weight to 256). It runs your calibration images through the network to observe the min/max activation values at each layer, then uses those to set quantisation scale factors. Poor calibration data → poor accuracy after quantisation.

## Phase 4: Set Up the Hailo Docker

### 4.1 Install Docker

```bash
sudo apt-get update
sudo apt-get install docker.io
sudo usermod -aG docker $USER
# Log out and back in for group membership to take effect
docker run hello-world  # verify
```

### 4.2 Download the Docker image

1. Go to https://hailo.ai/developer-zone/ and create a free account.
2. Navigate to **Software Downloads → AI Software Suite → Linux → x86 → Docker**.
3. Download the Docker package (e.g., `hailo8_ai_sw_suite_2025-10_docker.zip`). This is ~15–20 GB.

> ⚠️ **DOWNLOAD GOTCHA:** Some downloads from the Developer Zone are stubs — tiny files (a few KB) instead of the real image. The real `.tar.gz` is 5–10 GB. If it's under 1 GB, re-download while fully logged in, or try a different browser.

### 4.3 Load and run the container

```bash
# Unzip if needed:
unzip hailo8_ai_sw_suite_2025-10_docker.zip

# Load the Docker image:
docker load -i hailo8_ai_sw_suite_2025-10.tar.gz

# Check the exact image name and tag:
docker images | grep hailo

# Run the container:
docker run -it \
    --name hailo_compile \
    -v ~/hailo_compile:/local/shared_with_docker \
    hailo8_ai_sw_suite_2025-10:1
```

> **Note:** The image name prefix is `hailo8_` (not just `hailo_`). Always verify with `docker images` after loading.

**What this command deliberately omits:**
- `--privileged` — not needed (no device access)
- `-v /dev:/dev` — not needed (no hardware to pass through)
- `--gpus` — not needed (DFC runs on CPU)
- `--rm` — deliberately omitted so the container persists (re-enter with `docker start -i hailo_compile`)

### 4.4 Fix volume permissions (required!)

The `hailo` user inside the container runs as **UID 10642**, not your host UID (typically 1000). Without fixing this, the container can read your mounted files but **cannot write** to the shared volume.

```bash
# Exit the container first (type 'exit'), then on the host:
sudo chown -R 10642:10600 ~/hailo_compile

# Re-enter the container:
docker start -i hailo_compile

# Verify write access inside the container:
touch /local/shared_with_docker/test_write
rm /local/shared_with_docker/test_write
```

> ⚠️ **If you skip this step**, compilation may fail or the output HEF may not be accessible from the host.

### 4.5 Verify the environment

```bash
hailo -h              # DFC CLI
hailomz --help        # Model Zoo CLI
ls /local/shared_with_docker/models/    # your ONNX
ls /local/shared_with_docker/calib/ | wc -l   # calibration images
```

> You will see `[info] PCIe: No Hailo PCIe device was found` — **this is normal.** The DFC does not need a physical device.

## Phase 4.5: Smoke Test with Stock YOLOv8n (recommended)

Before compiling your custom model, verify the toolchain with the stock pretrained YOLOv8n.

### Download COCO calibration images

The Model Zoo's automatic calibration download may fail silently. Download manually:

```bash
cd /local/shared_with_docker
mkdir -p calib_coco && cd calib_coco
wget http://images.cocodataset.org/zips/val2017.zip
unzip val2017.zip && mv val2017/* . && rmdir val2017 && rm val2017.zip
```

### Compile

```bash
cd /local/shared_with_docker

hailomz compile yolov8n \
    --calib-path /local/shared_with_docker/calib_coco \
    --yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \
    --hw-arch hailo8l
```

> If the `--yaml` path doesn't work, find it: `find /local/workspace/hailo_model_zoo -name "yolov8n.yaml"`

This takes ~15–30 minutes on a modern CPU. You'll see `[info] No GPU chosen` — this is normal. Output: `yolov8n.hef` in your current directory.

## Phase 5: Compile Your Custom Model

```bash
cd /local/shared_with_docker

hailomz compile yolov8n \
    --ckpt /local/shared_with_docker/models/best.onnx \
    --calib-path /local/shared_with_docker/calib/ \
    --yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \
    --hw-arch hailo8l \
    --classes <YOUR_CLASS_COUNT>
```

**Key flags:**

| Flag | Purpose |
|------|---------|
| `yolov8n` / `yolov8s` | Selects the Model Zoo config. Must match your trained model variant. |
| `--ckpt` | Path to your ONNX file inside the container. |
| `--calib-path` | Directory of calibration images. |
| `--yaml` | Full path to the Model Zoo YAML config. |
| `--hw-arch hailo8l` | **CRITICAL:** Targets the Hailo-8L (13 TOPS). `hailo8` = 26 TOPS — **not interchangeable.** |
| `--classes N` | Your custom class count. Overrides the default 80 (COCO). |
| `--performance` | Optional. Better FPS via NPU scheduling optimisation. Longer compile time. |

## Phase 6: Set Up the Pi 5 + Hailo-8L

```bash
# Update
sudo apt update && sudo apt full-upgrade -y && sudo reboot

# Enable PCIe Gen 3 (important for performance)
# Add to /boot/firmware/config.txt:
dtparam=pciex1_gen=3
# Reboot

# Install HailoRT
sudo apt install hailo-all -y && sudo reboot

# Verify
hailortcli fw-control identify
# Should show: Device: HAILO8L

# Install hailo-apps
git clone https://github.com/hailo-ai/hailo-apps.git
cd hailo-apps && sudo ./install.sh
```

## Phase 7: Deploy and Run

```bash
# Transfer HEF from workstation
scp ~/hailo_compile/models/yolov8n.hef pi@<pi_ip>:~/models/

# Create labels JSON (adjust to match your data.yaml class order)
# ~/models/labels.json: {"class_a": 0, "class_b": 1, ...}

# Test with video file
cd ~/hailo-apps && source setup_env.sh
python basic_pipelines/detection.py \
    --hef ~/models/yolov8n.hef \
    --labels-json ~/models/labels.json \
    --input /path/to/test_video.mp4

# Run on RTSP camera
python basic_pipelines/detection.py \
    --hef ~/models/yolov8n.hef \
    --labels-json ~/models/labels.json \
    --input "rtsp://user:pass@camera_ip:554/stream"
```

## Container Management

```bash
# Re-enter existing container:
docker start -i hailo_compile

# Delete and recreate:
docker rm hailo_compile
docker run -it --name hailo_compile \
    -v ~/hailo_compile:/local/shared_with_docker \
    hailo8_ai_sw_suite_2025-10:1
# (remember to fix permissions again if recreating)
```

## Troubleshooting

| Issue | Solution |
|-------|----------|
| Permission denied on mounted volume | UID mismatch. Run on host: `sudo chown -R 10642:10600 ~/hailo_compile` |
| Calibration data not found (FileNotFoundError for .tfrecord) | Model Zoo auto-download fails silently. Download COCO val2017 manually and use `--calib-path`. |
| `--yaml` "cfg file is missing" | Use full path. Find with: `find /local/workspace/hailo_model_zoo -name "yolov8n.yaml"` |
| "unsupported op" during compilation | ONNX export must use `opset=11`. Don't include NMS. |
| `hailo8l` not recognised as `--hw-arch` | Older DFC version. Need suite 2025-10 or later. |
| Compilation runs out of memory | DFC needs 16 GB RAM minimum (32 GB recommended). |
| Low FPS on the Pi | Check PCIe Gen 3: `sudo lspci -vv \| grep -i LnkSta` → Speed 8GT/s. Try `--performance` flag. |
| Poor accuracy after quantisation | Use more calibration images (1000+). Ensure they're representative. Use `hailomz eval` to compare. |
| HEF not working on Pi | `--hw-arch` must match hardware: `hailo8l` = 13 TOPS, `hailo8` = 26 TOPS. NOT interchangeable. |
| "No GPU chosen" during compilation | Normal. DFC runs on CPU. |
| "No Hailo PCIe device found" | Normal. DFC doesn't need a physical device. |

## Useful Links

- [Hailo Developer Zone](https://hailo.ai/developer-zone/)
- [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo)
- [Hailo Apps (Pi 5)](https://github.com/hailo-ai/hailo-apps)
- [Hailo Community Forum](https://community.hailo.ai/)
- [Model Explorer](https://hailo.ai/products/hailo-software/model-explorer/)

---

*Tested April 2026 with Hailo AI SW Suite 2025-10, Docker 28.x on Linux Mint. Verify commands against current Hailo documentation.*

*Author: [Tom Herbison](https://github.com/mi0iou) — part of the [AI Security Camera](https://github.com/mi0iou/ai-security-camera) project.*