OpenNVR is an open-source (AGPL) NVR that records and streams from any ONVIF or RTSP camera and runs the AI model you choose behind them: YOLO detection, pose, face recognition, plate OCR, open-vocabulary detection, and a vision-language agent that can look at a scene and reason about it. It runs on hardware you own, air-gapped if you need, with every inference audited.
Version 0.1.10 is the first release with hardware acceleration flavours, and this post is about the one I have numbers for: Qualcomm's Hexagon NPU.
Arduino Ventuno Q: Qualcomm Dragonwing IQ-8275 (QCS8275), eight ARM cores, a Hexagon NPU, Ubuntu 24.04. Small, fanless, the kind of thing that sits behind a few cameras in a shop or a plant.
All measured on that board with the published container images. Full windows, settle times and caveats are in the benchmark doc.
Tier-0 detector (YOLO, one frame):
| Device | ms per frame |
|---|---|
| CPU, 8 threads | 164 |
| Hexagon NPU, fp16 | 8.6 |
In the full stack (streams, decode, tracking, the API, all on the same box):
| Device | Cameras at 2 fps, no drops | Host idle |
|---|---|---|
| CPU | 4 (10 cameras drop half their frames) | ~4 % |
| NPU | 16, 0 dropped | 46 % |
The adapters that sit behind the detector, end-to-end on one 1080p JPEG:
| Adapter | CPU | NPU |
|---|---|---|
| YOLO | 141 ms | 77 ms |
| Pose | 108 ms | 64 ms |
| Package detection | 100 ms | 60 ms |
| Face recognition (InsightFace) | 443 ms | 62 ms |
| Plate OCR | 248 ms | 62 ms |
The NPU's own share of an adapter call is about 9 ms. The rest is JPEG decode and preprocessing on a busy CPU, so the adapter path is now bounded by image handling, not inference.
Everything goes through ONNX Runtime's QNN execution provider, shipped as a plugin wheel (onnxruntime-qnn). There is no Qualcomm SDK to install on the host. The -qualcomm container gets the DSP device node, the DMA heap, and two host libraries bind-mounted, and that is it.
git clone https://github.com/open-nvr/open-nvr.git && cd open-nvr
./start.sh # OPENNVR_ACCEL=auto detects /dev/fastrpc-cdsp and picks the qualcomm flavour
OPENNVR_ACCEL=cpu|intel|qualcomm overrides the probe. A box with only a CPU gets exactly the images it got before.
Compiled graphs are cached, so the second start skips the ~6 s compile. The pipeline reports which provider ONNX Runtime really ran on, and a request for an accelerator that silently fell back to CPU shows up as requested != effective in the metrics instead of being hidden.
providers= list is silently ignored. It has to be attached by device with add_provider_for_devices. python:slim lacks libatomic1. The HTP backend fails to load and the session falls back to CPU with no error.up.
python -m detect_pipeline.bench --latency 200 --providers CPUExecutionProvider
python -m detect_pipeline.bench --latency 200 --providers QNNExecutionProvider:htp
It reports median, p90 and min per device, names the provider that really ran, and exits non-zero if the accelerator fell back.
NVIDIA, desktop GPUs and Jetson. If you have a Jetson, Rockchip or Hailo box you'd lend to a benchmark, I have one board and would like more.