OpenNVR 0.1.10: running an open-source NVR's AI on the Qualcomm Hexagon NPU

The OpenNVR project released version 0.1.10, the first with hardware-acceleration flavours, adding Qualcomm Hexagon NPU support via ONNX Runtime's QNN execution provider shipped as a plugin wheel. On an Arduino Ventuno Q (Dragonwing IQ-8275), YOLO detection dropped from 164 ms per frame on eight CPU threads to 8.6 ms on the NPU in fp16, and the full stack handled 16 cameras at 2 fps with no dropped frames versus 4 on CPU. The project reports the NPU's own share of an adapter call is about 9 ms, leaving JPEG decode and preprocessing as the bottleneck.

OpenNVR is an open-source AGPL NVR that records and streams from any ONVIF or RTSP camera and runs the AI model you choose behind them: YOLO detection, pose, face recognition, plate OCR, open-vocabulary detection, and a vision-language agent that can look at a scene and reason about it. It runs on hardware you own, air-gapped if you need, with every inference audited. Version 0.1.10 is the first release with hardware acceleration flavours, and this post is about the one I have numbers for: Qualcomm's Hexagon NPU. Arduino Ventuno Q: Qualcomm Dragonwing IQ-8275 QCS8275 , eight ARM cores, a Hexagon NPU, Ubuntu 24.04. Small, fanless, the kind of thing that sits behind a few cameras in a shop or a plant. All measured on that board with the published container images. Full windows, settle times and caveats are in the benchmark doc https://github.com/open-nvr/open-nvr/blob/main/bench/ventuno-q-benchmark-2026-10-30.md . Tier-0 detector YOLO, one frame : | Device | ms per frame | |---|---| | CPU, 8 threads | 164 | | Hexagon NPU, fp16 | 8.6 | In the full stack streams, decode, tracking, the API, all on the same box : | Device | Cameras at 2 fps, no drops | Host idle | |---|---|---| | CPU | 4 10 cameras drop half their frames | ~4 % | | NPU | 16, 0 dropped | 46 % | The adapters that sit behind the detector, end-to-end on one 1080p JPEG: | Adapter | CPU | NPU | |---|---|---| | YOLO | 141 ms | 77 ms | | Pose | 108 ms | 64 ms | | Package detection | 100 ms | 60 ms | | Face recognition InsightFace | 443 ms | 62 ms | | Plate OCR | 248 ms | 62 ms | The NPU's own share of an adapter call is about 9 ms. The rest is JPEG decode and preprocessing on a busy CPU, so the adapter path is now bounded by image handling, not inference. Everything goes through ONNX Runtime's QNN execution provider, shipped as a plugin wheel onnxruntime-qnn . There is no Qualcomm SDK to install on the host. The -qualcomm container gets the DSP device node, the DMA heap, and two host libraries bind-mounted, and that is it. git clone https://github.com/open-nvr/open-nvr.git && cd open-nvr ./start.sh OPENNVR ACCEL=auto detects /dev/fastrpc-cdsp and picks the qualcomm flavour OPENNVR ACCEL=cpu|intel|qualcomm overrides the probe. A box with only a CPU gets exactly the images it got before. Compiled graphs are cached, so the second start skips the ~6 s compile. The pipeline reports which provider ONNX Runtime really ran on, and a request for an accelerator that silently fell back to CPU shows up as requested = effective in the metrics instead of being hidden. providers= list is silently ignored. It has to be attached by device with add provider for devices . python:slim lacks libatomic1 . The HTP backend fails to load and the session falls back to CPU with no error. up . python -m detect pipeline.bench --latency 200 --providers CPUExecutionProvider python -m detect pipeline.bench --latency 200 --providers QNNExecutionProvider:htp It reports median, p90 and min per device, names the provider that really ran, and exits non-zero if the accelerator fell back. NVIDIA, desktop GPUs and Jetson. If you have a Jetson, Rockchip or Hailo box you'd lend to a benchmark, I have one board and would like more.