A practical guide for compiling custom-trained YOLOv8 models into HEF files using the official Hailo AI Software Suite Docker image — without needing Ubuntu, a Hailo PCIe device, or nvidia-docker on your host.
Target: Raspberry Pi 5 + Hailo-8L (13 TOPS) AI HAT+
Pipeline: .pt → .onnx → .har → .hef → deploy on Pi 5
Tested with: Hailo AI SW Suite 2025-10, Docker 28.x, Linux Mint (should work on any Linux with Docker)
Why This Guide Exists #
The official Hailo documentation assumes Ubuntu and provides a shell script (hailo_ai_sw_suite_docker_run.sh) that probes for PCIe drivers and nvidia-docker before launching the container. If you're compiling on a workstation that doesn't have a physical Hailo device installed — which is most people's setup — that script causes unnecessary problems. This guide bypasses it entirely with a direct docker run command.
What you need on the host:
- Docker (that's it)
- 16 GB+ RAM (32 GB recommended)
- 25 GB+ free disk space
- An NVIDIA GPU for training (not needed for compilation — the DFC runs on CPU)
What you do NOT need on the host:
- Ubuntu
- Hailo PCIe driver
- HailoRT
- nvidia-docker / NVIDIA Container Toolkit
- A physical Hailo device
Phase 1: Train Your Model #
Standard Ultralytics training — nothing Hailo-specific here:
from ultralytics import YOLO
model = YOLO('yolov8n.pt') # or yolov8s.pt
model.train(
data='your_dataset.yaml',
epochs=100,
imgsz=640,
batch=16,
device=0
)
Phase 2: Export to ONNX #
from ultralytics import YOLO
model = YOLO('runs/detect/train/weights/best.pt')
model.export(format='onnx', imgsz=640, opset=11, simplify=True)
Critical notes:
opset=11— Required by the Hailo DFC. Higher opset versions cause parsing failures.imgsz=640— Must match your training input size.simplify=True— Runs onnx-simplifier to remove redundant ops.- Do NOT add NMS to the ONNX — the Hailo pipeline handles NMS on the host CPU during inference.
Phase 3: Prepare Calibration Images #
The DFC needs representative images from your dataset to calibrate INT8 quantisation.
mkdir -p ~/hailo_compile/models
mkdir -p ~/hailo_compile/calib
cp runs/detect/train/weights/best.onnx ~/hailo_compile/models/
ls /path/to/dataset/train/images/ | shuf -n 1000 | \
xargs -I{} cp /path/to/dataset/train/images/{} ~/hailo_compile/calib/
- 256 minimum, 1000+ recommended.
- Images do NOT need to be pre-resized — the compiler handles resizing.
- Should be representative of real-world deployment conditions.
- Just raw images — no labels or annotations needed.
Why calibration matters: The DFC converts your model from FP32 to INT8 (from ~4 billion possible values per weight to 256). It runs your calibration images through the network to observe the min/max activation values at each layer, then uses those to set quantisation scale factors. Poor calibration data → poor accuracy after quantisation.
Phase 4: Set Up the Hailo Docker #
4.1 Install Docker
sudo apt-get update
sudo apt-get install docker.io
sudo usermod -aG docker $USER
docker run hello-world # verify
4.2 Download the Docker image
- Go to https://hailo.ai/developer-zone/ and create a free account.
- Navigate to Software Downloads → AI Software Suite → Linux → x86 → Docker.
- Download the Docker package (e.g.,
hailo8_ai_sw_suite_2025-10_docker.zip). This is ~15–20 GB.
⚠️ DOWNLOAD GOTCHA: Some downloads from the Developer Zone are stubs — tiny files (a few KB) instead of the real image. The real
.tar.gzis 5–10 GB. If it's under 1 GB, re-download while fully logged in, or try a different browser.
4.3 Load and run the container
unzip hailo8_ai_sw_suite_2025-10_docker.zip
docker load -i hailo8_ai_sw_suite_2025-10.tar.gz
docker images | grep hailo
docker run -it \
--name hailo_compile \
-v ~/hailo_compile:/local/shared_with_docker \
hailo8_ai_sw_suite_2025-10:1
Note: The image name prefix is
hailo8_(not justhailo_). Always verify withdocker imagesafter .
What this command deliberately omits:
--privileged— not needed (no device access)-v /dev:/dev— not needed (no hardware to pass through)--gpus— not needed (DFC runs on CPU)--rm— deliberately omitted so the container persists (re-enter withdocker start -i hailo_compile)
4.4 Fix volume permissions (required!)
The hailo user inside the container runs as UID 10642, not your host UID (typically 1000). Without fixing this, the container can read your mounted files but cannot write to the shared volume.
sudo chown -R 10642:10600 ~/hailo_compile
docker start -i hailo_compile
touch /local/shared_with_docker/test_write
rm /local/shared_with_docker/test_write
⚠️ If you skip this step, compilation may fail or the output HEF may not be accessible from the host.
4.5 Verify the environment
hailo -h # DFC CLI
hailomz --help # Model Zoo CLI
ls /local/shared_with_docker/models/ # your ONNX
ls /local/shared_with_docker/calib/ | wc -l # calibration images
You will see
[info] PCIe: No Hailo PCIe device was found— this is normal. The DFC does not need a physical device.
Phase 4.5: Smoke Test with Stock YOLOv8n (recommended) #
Before compiling your custom model, verify the toolchain with the stock pretrained YOLOv8n.
Download COCO calibration images
The Model Zoo's automatic calibration download may fail silently. Download manually:
cd /local/shared_with_docker
mkdir -p calib_coco && cd calib_coco
wget http://images.cocodataset.org/zips/val2017.zip
unzip val2017.zip && mv val2017/* . && rmdir val2017 && rm val2017.zip
Compile
cd /local/shared_with_docker
hailomz compile yolov8n \
--calib-path /local/shared_with_docker/calib_coco \
--yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \
--hw-arch hailo8l
If the
--yamlpath doesn't work, find it:find /local/workspace/hailo_model_zoo -name "yolov8n.yaml"
This takes ~15–30 minutes on a modern CPU. You'll see [info] No GPU chosen — this is normal. Output: yolov8n.hef in your current directory.
Phase 5: Compile Your Custom Model #
cd /local/shared_with_docker
hailomz compile yolov8n \
--ckpt /local/shared_with_docker/models/best.onnx \
--calib-path /local/shared_with_docker/calib/ \
--yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \
--hw-arch hailo8l \
--classes <YOUR_CLASS_COUNT>
Key flags:
| Flag | Purpose |
|---|---|
yolov8n / yolov8s |
Selects the Model Zoo config. Must match your trained model variant. |
--ckpt |
Path to your ONNX file inside the container. |
--calib-path |
Directory of calibration images. |
--yaml |
Full path to the Model Zoo YAML config. |
--hw-arch hailo8l |
CRITICAL: Targets the Hailo-8L (13 TOPS). hailo8 = 26 TOPS — not interchangeable. |
--classes N |
Your custom class count. Overrides the default 80 (COCO). |
--performance |
Optional. Better FPS via NPU scheduling optimisation. Longer compile time. |
Phase 6: Set Up the Pi 5 + Hailo-8L #
sudo apt update && sudo apt full-upgrade -y && sudo reboot
dtparam=pciex1_gen=3
sudo apt install hailo-all -y && sudo reboot
hailortcli fw-control identify
git clone https://github.com/hailo-ai/hailo-apps.git
cd hailo-apps && sudo ./install.sh
Phase 7: Deploy and Run #
scp ~/hailo_compile/models/yolov8n.hef pi@<pi_ip>:~/models/
cd ~/hailo-apps && source setup_env.sh
python basic_pipelines/detection.py \
--hef ~/models/yolov8n.hef \
--labels-json ~/models/labels.json \
--input /path/to/test_video.mp4
python basic_pipelines/detection.py \
--hef ~/models/yolov8n.hef \
--labels-json ~/models/labels.json \
--input "rtsp://user:pass@camera_ip:554/stream"
Container Management #
docker start -i hailo_compile
docker rm hailo_compile
docker run -it --name hailo_compile \
-v ~/hailo_compile:/local/shared_with_docker \
hailo8_ai_sw_suite_2025-10:1
Troubleshooting #
| Issue | Solution |
|---|---|
| Permission denied on mounted volume | UID mismatch. Run on host: sudo chown -R 10642:10600 ~/hailo_compile |
| Calibration data not found (FileNotFoundError for .tfrecord) | Model Zoo auto-download fails silently. Download COCO val2017 manually and use --calib-path. |
--yaml "cfg file is missing" |
Use full path. Find with: find /local/workspace/hailo_model_zoo -name "yolov8n.yaml" |
| "unsupported op" during compilation | ONNX export must use opset=11. Don't include NMS. |
hailo8l not recognised as --hw-arch |
Older DFC version. Need suite 2025-10 or later. |
| Compilation runs out of memory | DFC needs 16 GB RAM minimum (32 GB recommended). |
| Low FPS on the Pi | Check PCIe Gen 3: sudo lspci -vv | grep -i LnkSta → Speed 8GT/s. Try --performance flag. |
| Poor accuracy after quantisation | Use more calibration images (1000+). Ensure they're representative. Use hailomz eval to compare. |
| HEF not working on Pi | --hw-arch must match hardware: hailo8l = 13 TOPS, hailo8 = 26 TOPS. NOT interchangeable. |
| "No GPU chosen" during compilation | Normal. DFC runs on CPU. |
| "No Hailo PCIe device found" | Normal. DFC doesn't need a physical device. |
Useful Links #
Tested April 2026 with Hailo AI SW Suite 2025-10, Docker 28.x on Linux Mint. Verify commands against current Hailo documentation.
Author: Tom Herbison — part of the AI Security Camera project.