{"slug": "deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l", "title": "Deploying Custom YOLOv8 on Raspberry Pi 5 + Hailo-8L", "summary": "Practical guide for compiling custom-trained YOLOv8 models into HEF files for deployment on a Raspberry Pi 5 with a Hailo-8L AI HAT+, using the official Hailo AI Software Suite Docker container. The guide bypasses the standard Hailo setup script to allow compilation on any Linux host with Docker, without requiring Ubuntu, a physical Hailo device, or an NVIDIA GPU. It details the full pipeline from exporting a YOLOv8 model to ONNX format, preparing calibration images for INT8 quantization, and running the Docker container to generate the final HEF file.", "body_md": "# Compiling Custom YOLOv8 HEFs on Non-Ubuntu Linux Using the Hailo Docker (No Shell Script)\n\n**A practical guide for compiling custom-trained YOLOv8 models into HEF files using the official Hailo AI Software Suite Docker image — without needing Ubuntu, a Hailo PCIe device, or nvidia-docker on your host.**\n\n**Target:** Raspberry Pi 5 + Hailo-8L (13 TOPS) AI HAT+  \n**Pipeline:** `.pt → .onnx → .har → .hef → deploy on Pi 5`  \n**Tested with:** Hailo AI SW Suite 2025-10, Docker 28.x, Linux Mint (should work on any Linux with Docker)\n\n---\n\n## Why This Guide Exists\n\nThe official Hailo documentation assumes Ubuntu and provides a shell script (`hailo_ai_sw_suite_docker_run.sh`) that probes for PCIe drivers and nvidia-docker before launching the container. If you're compiling on a workstation that doesn't have a physical Hailo device installed — which is most people's setup — that script causes unnecessary problems. This guide bypasses it entirely with a direct `docker run` command.\n\n**What you need on the host:**\n- Docker (that's it)\n- 16 GB+ RAM (32 GB recommended)\n- 25 GB+ free disk space\n- An NVIDIA GPU for training (not needed for compilation — the DFC runs on CPU)\n\n**What you do NOT need on the host:**\n- Ubuntu\n- Hailo PCIe driver\n- HailoRT\n- nvidia-docker / NVIDIA Container Toolkit\n- A physical Hailo device\n\n---\n\n## Phase 1: Train Your Model\n\nStandard Ultralytics training — nothing Hailo-specific here:\n\n```python\nfrom ultralytics import YOLO\n\nmodel = YOLO('yolov8n.pt')  # or yolov8s.pt\nmodel.train(\n    data='your_dataset.yaml',\n    epochs=100,\n    imgsz=640,\n    batch=16,\n    device=0\n)\n# Result: runs/detect/train/weights/best.pt\n```\n\n## Phase 2: Export to ONNX\n\n```python\nfrom ultralytics import YOLO\n\nmodel = YOLO('runs/detect/train/weights/best.pt')\nmodel.export(format='onnx', imgsz=640, opset=11, simplify=True)\n# Result: runs/detect/train/weights/best.onnx\n```\n\n**Critical notes:**\n- **`opset=11`** — Required by the Hailo DFC. Higher opset versions cause parsing failures.\n- **`imgsz=640`** — Must match your training input size.\n- **`simplify=True`** — Runs onnx-simplifier to remove redundant ops.\n- **Do NOT add NMS** to the ONNX — the Hailo pipeline handles NMS on the host CPU during inference.\n\n## Phase 3: Prepare Calibration Images\n\nThe DFC needs representative images from your dataset to calibrate INT8 quantisation.\n\n```bash\nmkdir -p ~/hailo_compile/models\nmkdir -p ~/hailo_compile/calib\n\ncp runs/detect/train/weights/best.onnx ~/hailo_compile/models/\n\n# Copy a random subset of training images (just images, no labels):\nls /path/to/dataset/train/images/ | shuf -n 1000 | \\\n    xargs -I{} cp /path/to/dataset/train/images/{} ~/hailo_compile/calib/\n```\n\n- **256 minimum, 1000+ recommended.**\n- Images do NOT need to be pre-resized — the compiler handles resizing.\n- Should be representative of real-world deployment conditions.\n- Just raw images — no labels or annotations needed.\n\n**Why calibration matters:** The DFC converts your model from FP32 to INT8 (from ~4 billion possible values per weight to 256). It runs your calibration images through the network to observe the min/max activation values at each layer, then uses those to set quantisation scale factors. Poor calibration data → poor accuracy after quantisation.\n\n## Phase 4: Set Up the Hailo Docker\n\n### 4.1 Install Docker\n\n```bash\nsudo apt-get update\nsudo apt-get install docker.io\nsudo usermod -aG docker $USER\n# Log out and back in for group membership to take effect\ndocker run hello-world  # verify\n```\n\n### 4.2 Download the Docker image\n\n1. Go to https://hailo.ai/developer-zone/ and create a free account.\n2. Navigate to **Software Downloads → AI Software Suite → Linux → x86 → Docker**.\n3. Download the Docker package (e.g., `hailo8_ai_sw_suite_2025-10_docker.zip`). This is ~15–20 GB.\n\n> ⚠️ **DOWNLOAD GOTCHA:** Some downloads from the Developer Zone are stubs — tiny files (a few KB) instead of the real image. The real `.tar.gz` is 5–10 GB. If it's under 1 GB, re-download while fully logged in, or try a different browser.\n\n### 4.3 Load and run the container\n\n```bash\n# Unzip if needed:\nunzip hailo8_ai_sw_suite_2025-10_docker.zip\n\n# Load the Docker image:\ndocker load -i hailo8_ai_sw_suite_2025-10.tar.gz\n\n# Check the exact image name and tag:\ndocker images | grep hailo\n\n# Run the container:\ndocker run -it \\\n    --name hailo_compile \\\n    -v ~/hailo_compile:/local/shared_with_docker \\\n    hailo8_ai_sw_suite_2025-10:1\n```\n\n> **Note:** The image name prefix is `hailo8_` (not just `hailo_`). Always verify with `docker images` after loading.\n\n**What this command deliberately omits:**\n- `--privileged` — not needed (no device access)\n- `-v /dev:/dev` — not needed (no hardware to pass through)\n- `--gpus` — not needed (DFC runs on CPU)\n- `--rm` — deliberately omitted so the container persists (re-enter with `docker start -i hailo_compile`)\n\n### 4.4 Fix volume permissions (required!)\n\nThe `hailo` user inside the container runs as **UID 10642**, not your host UID (typically 1000). Without fixing this, the container can read your mounted files but **cannot write** to the shared volume.\n\n```bash\n# Exit the container first (type 'exit'), then on the host:\nsudo chown -R 10642:10600 ~/hailo_compile\n\n# Re-enter the container:\ndocker start -i hailo_compile\n\n# Verify write access inside the container:\ntouch /local/shared_with_docker/test_write\nrm /local/shared_with_docker/test_write\n```\n\n> ⚠️ **If you skip this step**, compilation may fail or the output HEF may not be accessible from the host.\n\n### 4.5 Verify the environment\n\n```bash\nhailo -h              # DFC CLI\nhailomz --help        # Model Zoo CLI\nls /local/shared_with_docker/models/    # your ONNX\nls /local/shared_with_docker/calib/ | wc -l   # calibration images\n```\n\n> You will see `[info] PCIe: No Hailo PCIe device was found` — **this is normal.** The DFC does not need a physical device.\n\n## Phase 4.5: Smoke Test with Stock YOLOv8n (recommended)\n\nBefore compiling your custom model, verify the toolchain with the stock pretrained YOLOv8n.\n\n### Download COCO calibration images\n\nThe Model Zoo's automatic calibration download may fail silently. Download manually:\n\n```bash\ncd /local/shared_with_docker\nmkdir -p calib_coco && cd calib_coco\nwget http://images.cocodataset.org/zips/val2017.zip\nunzip val2017.zip && mv val2017/* . && rmdir val2017 && rm val2017.zip\n```\n\n### Compile\n\n```bash\ncd /local/shared_with_docker\n\nhailomz compile yolov8n \\\n    --calib-path /local/shared_with_docker/calib_coco \\\n    --yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \\\n    --hw-arch hailo8l\n```\n\n> If the `--yaml` path doesn't work, find it: `find /local/workspace/hailo_model_zoo -name \"yolov8n.yaml\"`\n\nThis takes ~15–30 minutes on a modern CPU. You'll see `[info] No GPU chosen` — this is normal. Output: `yolov8n.hef` in your current directory.\n\n## Phase 5: Compile Your Custom Model\n\n```bash\ncd /local/shared_with_docker\n\nhailomz compile yolov8n \\\n    --ckpt /local/shared_with_docker/models/best.onnx \\\n    --calib-path /local/shared_with_docker/calib/ \\\n    --yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8n.yaml \\\n    --hw-arch hailo8l \\\n    --classes <YOUR_CLASS_COUNT>\n```\n\n**Key flags:**\n\n| Flag | Purpose |\n|------|---------|\n| `yolov8n` / `yolov8s` | Selects the Model Zoo config. Must match your trained model variant. |\n| `--ckpt` | Path to your ONNX file inside the container. |\n| `--calib-path` | Directory of calibration images. |\n| `--yaml` | Full path to the Model Zoo YAML config. |\n| `--hw-arch hailo8l` | **CRITICAL:** Targets the Hailo-8L (13 TOPS). `hailo8` = 26 TOPS — **not interchangeable.** |\n| `--classes N` | Your custom class count. Overrides the default 80 (COCO). |\n| `--performance` | Optional. Better FPS via NPU scheduling optimisation. Longer compile time. |\n\n## Phase 6: Set Up the Pi 5 + Hailo-8L\n\n```bash\n# Update\nsudo apt update && sudo apt full-upgrade -y && sudo reboot\n\n# Enable PCIe Gen 3 (important for performance)\n# Add to /boot/firmware/config.txt:\ndtparam=pciex1_gen=3\n# Reboot\n\n# Install HailoRT\nsudo apt install hailo-all -y && sudo reboot\n\n# Verify\nhailortcli fw-control identify\n# Should show: Device: HAILO8L\n\n# Install hailo-apps\ngit clone https://github.com/hailo-ai/hailo-apps.git\ncd hailo-apps && sudo ./install.sh\n```\n\n## Phase 7: Deploy and Run\n\n```bash\n# Transfer HEF from workstation\nscp ~/hailo_compile/models/yolov8n.hef pi@<pi_ip>:~/models/\n\n# Create labels JSON (adjust to match your data.yaml class order)\n# ~/models/labels.json: {\"class_a\": 0, \"class_b\": 1, ...}\n\n# Test with video file\ncd ~/hailo-apps && source setup_env.sh\npython basic_pipelines/detection.py \\\n    --hef ~/models/yolov8n.hef \\\n    --labels-json ~/models/labels.json \\\n    --input /path/to/test_video.mp4\n\n# Run on RTSP camera\npython basic_pipelines/detection.py \\\n    --hef ~/models/yolov8n.hef \\\n    --labels-json ~/models/labels.json \\\n    --input \"rtsp://user:pass@camera_ip:554/stream\"\n```\n\n## Container Management\n\n```bash\n# Re-enter existing container:\ndocker start -i hailo_compile\n\n# Delete and recreate:\ndocker rm hailo_compile\ndocker run -it --name hailo_compile \\\n    -v ~/hailo_compile:/local/shared_with_docker \\\n    hailo8_ai_sw_suite_2025-10:1\n# (remember to fix permissions again if recreating)\n```\n\n## Troubleshooting\n\n| Issue | Solution |\n|-------|----------|\n| Permission denied on mounted volume | UID mismatch. Run on host: `sudo chown -R 10642:10600 ~/hailo_compile` |\n| Calibration data not found (FileNotFoundError for .tfrecord) | Model Zoo auto-download fails silently. Download COCO val2017 manually and use `--calib-path`. |\n| `--yaml` \"cfg file is missing\" | Use full path. Find with: `find /local/workspace/hailo_model_zoo -name \"yolov8n.yaml\"` |\n| \"unsupported op\" during compilation | ONNX export must use `opset=11`. Don't include NMS. |\n| `hailo8l` not recognised as `--hw-arch` | Older DFC version. Need suite 2025-10 or later. |\n| Compilation runs out of memory | DFC needs 16 GB RAM minimum (32 GB recommended). |\n| Low FPS on the Pi | Check PCIe Gen 3: `sudo lspci -vv \\| grep -i LnkSta` → Speed 8GT/s. Try `--performance` flag. |\n| Poor accuracy after quantisation | Use more calibration images (1000+). Ensure they're representative. Use `hailomz eval` to compare. |\n| HEF not working on Pi | `--hw-arch` must match hardware: `hailo8l` = 13 TOPS, `hailo8` = 26 TOPS. NOT interchangeable. |\n| \"No GPU chosen\" during compilation | Normal. DFC runs on CPU. |\n| \"No Hailo PCIe device found\" | Normal. DFC doesn't need a physical device. |\n\n## Useful Links\n\n- [Hailo Developer Zone](https://hailo.ai/developer-zone/)\n- [Hailo Model Zoo](https://github.com/hailo-ai/hailo_model_zoo)\n- [Hailo Apps (Pi 5)](https://github.com/hailo-ai/hailo-apps)\n- [Hailo Community Forum](https://community.hailo.ai/)\n- [Model Explorer](https://hailo.ai/products/hailo-software/model-explorer/)\n\n---\n\n*Tested April 2026 with Hailo AI SW Suite 2025-10, Docker 28.x on Linux Mint. Verify commands against current Hailo documentation.*\n\n*Author: [Tom Herbison](https://github.com/mi0iou) — part of the [AI Security Camera](https://github.com/mi0iou/ai-security-camera) project.*\n", "url": "https://wpnews.pro/news/deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l", "canonical_source": "https://gist.github.com/mi0iou/87c4b5315e4e0edee0437c7e04290cdb", "published_at": "2026-04-12 18:20:10+00:00", "updated_at": "2026-05-22 08:59:36.111122+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "hardware", "developer-tools", "robotics"], "entities": ["Raspberry Pi", "Hailo", "YOLOv8", "Ultralytics", "Docker", "Linux Mint", "NVIDIA"], "alternates": {"html": "https://wpnews.pro/news/deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l", "markdown": "https://wpnews.pro/news/deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l.md", "text": "https://wpnews.pro/news/deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l.txt", "jsonld": "https://wpnews.pro/news/deploying-custom-yolov8-on-raspberry-pi-5-hailo-8l.jsonld"}}