We Have AI at Home: Raspberry Pi AI Hat+ 2 Hands On

Tracking routes through a production hall, finding bottlenecks, and optimizing workflows, all locally without a single image leaving the premises: these are exactly the kinds of jobs the Raspberry Pi 5 with AI HAT+ 2 sets out to handle. For this hands-on test, a supermarket parking lot stands in for the factory floor. The Pi records almost 11,000 camera images, then detects vehicles and people locally, creates separate privacy-protected versions, and finally writes its own report with a local language model. The test shows what the Hailo-10H really accelerates, where the models and host CPU hit their limits, and why the Pi’s 16 GB of RAM may be the accelerator’s most important companion.

A complete local AI pipeline #

One August afternoon, my test setup is sitting in a supermarket parking lot: a Raspberry Pi 5 powered by a power bank, with a USB webcam looking across the parking spaces. In 36 minutes, the system saves 10,944 unmodified camera frames, first from the parking lot and then on the drive home. Back at home, the same Pi handles the analysis. The AI HAT+ 2 detects cars, people, trucks, and buses; a tracker follows them from frame to frame; a privacy stage pixelates heads and license-plate areas; and a local language model writes the prose for an HTML report, which the Pi also generates itself. Not one image or measurement leaves the device.

That is precisely the pipeline I wanted to test, because it answers the genuinely interesting question better than any synthetic benchmark: what does the AI accelerator actually handle, and what remains the Raspberry Pi’s job?

Setup and design decisions #

The test system consists of a Raspberry Pi 5 Model B with 16 GB of RAM — paid Amazon affiliate link (Rev 1.1) and a PCIe-connected AI HAT+ 2 — paid Amazon affiliate link, carrying a Hailo-10H with 8 GB of its own memory. Unlike pure vision accelerators, the Hailo-10H can also run generative models, including small language and vision-language models, and it gets an opportunity to do so later in the test. The operating system is Raspberry Pi OS Lite, specifically Debian 13.6 with Raspberry Pi kernel 6.18. I abandoned my original plan to use Ubuntu Server because the official Raspberry Pi and Hailo packages are tailored to Raspberry Pi’s own operating system, from the PCIe driver and camera stack to the example collections. The Pi runs entirely headless, I access it through SSH, and the detection pipelines send their video to a fakesink

, so they need no desktop at all.

The AI software comes from the official Hailo stack: HailoRT and firmware 5.1.1, TAPPAS Core 5.1.0, and the hailo-apps 26.3.0 Python package with its collection of examples. Models run on the Hailo-10H neither as ONNX files nor as PyTorch checkpoints. Instead, they use Hailo Executable Format, or HEF, which Hailo compiles for its accelerators and provides ready-made for many common models in its

[Model Zoo](https://github.com/hailo-ai/hailo_model_zoo). The field test used

[, a roughly 25 MB model trained on the standard](https://github.com/ultralytics/ultralytics)

yolov8m.hef

COCO classes. Detections with at least 25 percent confidence go into the database so that uncertain candidates remain available for later analysis. The evaluation and every image in this article use the stricter 50 percent threshold. Anyone who wants to use a custom model must convert it with the Hailo Dataflow Compiler, a process I return to later.

The camera was an Elgato Facecam — paid Amazon affiliate link. It appears to the Pi as a standard UVC device and supplies 1,920 × 1,080 MJPEG video at 30 frames per second, with automatic white balance and exposure.

The field test and data pipeline #

I deliberately separated capture and analysis for this test. In the field, the Pi does nothing but save unmodified original images. The computationally intensive Hailo analysis then runs on the same device afterward. This keeps the mobile phase simple and, more importantly, produces an immutable raw dataset that can later be used to compare as many models and pipelines as needed under identical conditions. Every benchmark in this article runs on exactly the same field images.

I chose the parking lot because it offered a variety of subjects in a small area: people, moving and parked cars, delivery vehicles, and shopping carts. The drive home added buses, traffic lights, and road signs.

Recording itself was reliable. A systemd service started capture automatically when power was connected and saved every sixth camera frame, giving 5 frames per second. It wrote the unmodified JPEG data straight from the MJPEG stream, without recompression, labels, or color correction. In parallel, the service logged timestamps, free storage, and the Pi’s system status. After 36 minutes and 7 seconds, the SD card held 10,944 originals totaling about 2.7 GB, and all 432 status samples were unremarkable. A configured minimum reserve of 10 GiB free space would have stopped recording cleanly before the card filled up. I excluded a short segment during which I moved the camera into the car for the drive home. In total, 10,244 images went into the analysis.

The data revealed a problem that any field pipeline needs to anticipate: 106 files were only 400 to 500 bytes long and contained no readable JPEG stream. They appeared unusually often in contiguous groups during the drive home. I could not prove the cause, and all logged status values remained normal throughout. Brief USB transfer disruptions caused by motion and vibration in the car are a plausible explanation. I retained the defective files and marked them as errors in the database. Anyone who monitors only AI output but not the integrity of the input data will miss gaps like these.

Batch analysis on the Hailo-10H processed the 10,244 frames in 544 seconds. That works out to 18.8 frames per second end to end, including JPEG reads, error handling, ByteTrack tracking for each object class, and SQLite writes. The combined detection stage, consisting of image reading, preprocessing, Hailo inference, and tracking, averaged 50.43 milliseconds per valid frame. Temperature hovered between 51 and just under 60 degrees Celsius, without any throttling.

I paid particular attention to the order of the privacy processing. The rule is simple: inference runs first on the unmodified original, the detection coordinates are stored next, only then does the pipeline create a working copy and pixelate people’s head regions and the typical license-plate zones on vehicles, and finally it draws the detection boxes as the topmost layer. This keeps pixelation from affecting detection and ensures that no box disappears beneath a pixel block. I built the first version in the wrong order and pixelated copies that already had annotations. One run across all 10,138 valid frames created the complete privacy-protected set in just over five minutes at 33.4 frames per second. The pixelation itself takes only about 0.3 milliseconds per image. Post-processing is dominated by JPEG decoding and encoding, at just over 12 milliseconds each.

I also tested the deliberately conservative pixelation heuristic with the installed SCRFD face detector on 1,000 comparison images. Of 59 detected face boxes, 98.3 percent lay inside the head regions that had already been pixelated, and every single one lay within a detected person box. A production version should still use the union of both methods for greater safety. Neither eliminates the need for manual visual review before publication, especially because the pixelated vehicle area is only an approximation rather than validated license-plate detection.

Is it fast enough for real time? #

Adding the measured stages gives a first indication of whether the complete system can operate in real time. The 50.4 millisecond detection stage plus 29.9 milliseconds of post-processing totals 80.4 milliseconds per frame, or 12.4 frames per second in theory. Because the two separate runs each decoded the same JPEG again, while an integrated pipeline would need to read the camera frame only once, a cautious projection lowers the figure to 67.9 milliseconds, or 14.7 FPS. At the planned 5 FPS, every frame has a 200 millisecond budget, leaving plenty of headroom.

A replay test checked whether that calculation held in practice. One pipeline played back 1,000 stored field images at exactly 5 FPS and handled reading, rotation, preprocessing, Hailo inference, detection processing, tracking, pixelation, overlay, JPEG output, and database logging for each frame in a single pass. The result was 1,000 images in 199.883 seconds, or 5.003 FPS, with an average total latency of 67.5 milliseconds. The median was 65.7 milliseconds and the 95th percentile was 80 milliseconds. Of the 1,000 frames, 999 met their 200 millisecond deadline. The individual stages are revealing. Hailo inference was remarkably consistent, averaging 25.5 milliseconds with a 95th percentile of 26.0. The sole missed deadline came from a one-off JPEG write to the SD card that took 254 milliseconds, and the following frame caught up immediately. A permanently reliable live system should therefore move image output into an asynchronous write queue, decoupled from the inference path. The replay test provides a reproducible demonstration that every processing stage can run together on the Pi while maintaining the target rate.

What does the system detect? #

The SQLite database ultimately contains 107,914 detection events at or above the 25 percent storage threshold. That figure needs context, because a detection event is not a unique object. A parked car is detected again across thousands of frames and counts anew every time. These values show detection activity and class distribution, not the number of real vehicles or people.

At the 50 percent evaluation threshold, cars dominate as expected with 60,665 events, followed by 5,199 people, 2,207 trucks, and 1,041 buses. The drive home contributed another 135 traffic-light events and 13 stop-sign events, plus 55 bicycle events and 3 motorcycle events across both segments. The highest confidence for an individual detection was 97.4 percent for a bus, 96.6 for a truck, 96.3 for a person, and 95.9 for a car. Per-class ByteTrack tracking provides a cautious approximation of actual objects. The parking lot produced 361 stable car track segments, 157 person segments, 104 truck segments, and 12 bus segments with at least five observations. The drive home produced 242 car, 29 truck, 16 person, and 5 bus segments. These are not exact counts either. Occlusion, camera motion, and ID changes can split one real object into multiple segments. Reliable numbers require tracking, scene logic, and above all a precise definition of what is being counted.

And then there was the shopping cart. It simply does not appear in the results because YOLOv8m was trained on the standard COCO classes, and that catalog has no shopping-cart class. In one sample image, the model casually classified a cart as a bicycle. This may be the test’s most important substantive finding: more computing power cannot create a class the model never learned.

How much faster is the HAT than the CPU? #

A direct comparison quantifies the accelerator’s main benefit. YOLOv8m processed exactly the same 200 field images once as a compiled HEF file on the Hailo-10H and once as a full-precision ONNX model from the Hailo Model Zoo, using ONNX Runtime on all four Pi CPU cores. Both used identical 640-pixel letterbox preprocessing. Because HEF and ONNX are quantized or compiled for their respective platforms, this compares the same model family rather than bit-identical weights.

The difference is substantial. Pure inference averaged 25.541 milliseconds on the HAT and 964.358 milliseconds on the CPU, a factor of 37.76. End to end, including image processing, the HAT achieved 19.890 frames per second while the CPU managed 1.013, a factor of 19.64. Reassuringly, both paths returned almost the same output volume, with 1,614 versus 1,611 detections. That is a useful plausibility check, but without manually annotated ground truth it does not prove accuracy. In practical terms, the CPU alone is already too slow for the planned 5 FPS pipeline. The HAT offers four times that throughput and leaves the CPU free for tracking, privacy processing, storage, and application logic.

Trying to improve the CPU result with a larger model only reinforces the point. According to the Ultralytics reference, YOLOv8x is slightly more accurate on COCO than YOLOv8m, but it fell to 0.337 frames per second on the Pi’s CPU, reached 83.7 degrees Celsius, and triggered soft thermal throttling. The 0x80008

status from vcgencmd get_throttled

reveals two things: the soft temperature limit was still active at the end of the measurement, and it had been reached at least once during the run. After the Pi cooled down, only the historical event bit remained set, as expected, until the next restart. Memory use was just over 700 MB for the process, far below the Pi’s 16 GB. For computer vision on the Pi, compute performance rather than RAM is the bottleneck, and better cooling does not fundamentally change that.

That leaves endurance. After a fresh restart, a 30-minute maximum-load test ran the real HAT pipeline without , including JPEG reading, rotation, and post-processing. It processed 37,004 images at an average of 20.557 FPS, with zero errors, a maximum temperature of 61.15 degrees Celsius, and throttled=0x0

in all 1,786 monitoring samples. Mean inference time was 25.4 milliseconds, with a 95th percentile of 25.8, while the slowest individual image took 97 milliseconds. After only a few minutes, temperature settled at 59 to 60 degrees and rose no further. Throughput was just 0.16 percent lower than a short two-minute reference, so there is no sign of long-term performance decline. One other detail stands out: the complete vision pipeline used at most 140 MB of process memory, leaving almost all of the Pi’s RAM free. Sustained load on the HAT is utterly uneventful for the Pi, in the best possible way.

Model choice and the limits of object detection #

A faster model is not automatically a better one. A comparison between YOLOv8m and the much smaller YOLOv6n on the same 1,000 images, split evenly between the parking lot and the drive home, made that clear. The 3.9 MB model almost halves Hailo inference time, taking 12.7 rather than 25.5 milliseconds, and reaches 27.6 rather than 19.9 FPS end to end. At the 50 percent threshold, however, it found noticeably fewer examples of the classes that mattered in this test: 336 instead of 392 person detections, 86 instead of 204 trucks, and only 4 instead of 249 buses.

An agreement analysis shows how similar the two models nevertheless are. Both outputs contained 4,645 pairs of same-class boxes with at least 50 percent overlap. That gives 80.2 percent of the YOLOv8m boxes a counterpart in YOLOv6n and 94.3 percent of the YOLOv6n boxes a counterpart in YOLOv8m, consistent with the larger model producing more high-confidence boxes overall. Without ground truth, this still does not establish a general accuracy ranking, particularly because confidence values are not calibrated between models. At the lower 25 percent threshold, YOLOv6n actually produced more boxes overall than YOLOv8m. The choice depends on the application. A need for maximum headroom and several parallel streams favors the smaller model, while YOLOv8m found more of the rarer classes in this test. The Hailo installation also includes pose and segmentation models, but I did not test them here.

For the shopping-cart problem, I tried a detour through the installed Hailo CLIP pipeline, which compares image crops against freely worded terms. After some tinkering, a small proof of concept worked. The generation scripts mentioned in the README were missing from the application directory, but the tokenizer, token-lookup, and projection files were available in the system resources. SHA-256 checks showed that they matched an independent extraction from the official OpenAI CLIP weights. A control test confirmed the text pipeline with a cosine similarity of 0.9919 against Hailo’s supplied sample embedding. In the experiment itself, six manually selected crops from a parking-lot frame each competed against six terms. CLIP classified the genuine shopping-cart crop correctly. “Shopping cart” won with a cosine similarity of 0.312, ahead of “car” at 0.262. Overall, however, only three of the six crops were correct at Top-1. Wide scene crops with distracting context failed: the building crop also contained vehicles, while the road crop included part of my own car. The method was certainly fast enough, with the image pipeline processing 23.3 crops per second and a text prompt costing 0.43 seconds once. One quirk in the packaged GStreamer pipeline did consume debugging time. At the end of a finite input, it held back the final buffer. Only an appended flush frame, which I did not evaluate, delivered all six image embeddings to the callback.

Most importantly, CLIP does not find an object’s bounding box itself. It only classifies the crop it is given. This does not produce automatic shopping-cart counts. That would require a candidate-generation stage or, better still, a specialized detector trained outside the Pi and translated into an HEF with the Hailo Dataflow Compiler.

Second scenario: local language and vision models #

The second main scenario was designed to find out how well the Pi and HAT could work as a local report writer. The architecture mattered to me. The language model receives neither images nor the raw database. Instead, it gets a deterministically generated fact list from the SQLite analysis. Numbers, tables, and charts are created without AI. The model supplies only the prose, and an automated fact check then verifies every text against the provided data.

The Hailo Apps provide Qwen2.5-1.5B-Instruct for the Hailo-10H, with a context window of 2,048 tokens. The supplied down initially failed with a 404 because, despite the installed version being 5.1.1, it pointed to a 5.1.0 path. The 2.2 GiB HEF file was available at the correct official path, and HailoRT confirmed its compatibility. The model then started pleasantly quickly on the accelerator. The virtual device initialized in 0.045 seconds and the model loaded in 8.7 seconds. After that, time to first token averaged 0.7 seconds, with output at 6.9 tokens per second. The host process used only about 173 MB of RAM because the model resides in the Hailo-10H’s memory. Unfortunately, the small model’s German report was unusable. It repeated parts of the instruction, ignored formatting requirements, and invented percentages of 96.7 and 10.6 that appeared nowhere in the measurements. The fact validator rejected the draft, which is exactly why it exists. Acceleration does not guarantee reliable text.

The cross-check made the result even more interesting. The same 1.5B model in GGUF format, running through llama.cpp on the Pi’s CPU, managed 12.0 tokens per second. That was 1.74 times faster than the HAT, although its German prose was similarly weak, CPU load was much higher, and the process used just over 2.8 GB of memory instead of 173 MB. The AI HAT+ 2 is not a universal AI turbocharger. With this small LLM, its advantage lies not in throughput but in off the work and placing almost no burden on the host.

This is where the Pi’s 16 GB comes into play. It can accommodate Qwen2.5-7B-Instruct in Q4_K_M quantization, whose files occupy about 4.36 GiB and whose process peaked at roughly 8.67 GiB in operation, without swap or throttling. Prompt processing ran at 11.5 tokens per second and text generation at 2.66. Each of the four report sections therefore spent a little over 21 seconds in the prompt phase alone. At about six and a half minutes for all four sections, this is no real-time tool. The 7B text did pass the fact check, however. It invented no numbers and correctly described the detections as repeated observations. I deliberately did not polish the model’s prose in the final report, so readers can see what the local model actually produces. Its language is awkward in places.

The finished HTML report, generated entirely on the Pi, combines a key-metrics section, class distribution, a performance chart, and two full-resolution privacy-protected images with locally generated prose, the machine-readable fact list, and a separate validation log. The presentation worked too. In my test, the report displayed correctly on both a desktop and a narrow 390-pixel mobile view without horizontal scrolling. This suggests a robust pattern for local AI reports: deterministic data and charts, an LLM used only as a writing layer, and automated verification afterward.

As a third example, I tested Qwen2-VL-2B on the HAT, a vision-language model that describes images directly in text. Its 2.2 GiB HEF file also had to be downloaded from the correct version path instead of through the broken down. A control run with Hailo’s supplied English sample image worked immediately. I then tested eight representative field images with a fixed seed and an identical prompt. Technically, the model ran well. Once warmed up, it was ready in just under 10 seconds. Time to the first text segment averaged 1.23 seconds, output reached 8.3 text segments per second, and total time was around 14 seconds per image, all with minimal host load.

In terms of content, it is useful only for experiments at this stage. The official application’s square preprocessing scales down the 16:9 image and center-crops it to 336 × 336 pixels. Just 56.28 percent of the scaled image area reaches the model, so a wide-angle scene loses almost half of its information. German prompts consistently produced broken answers. English made the descriptions more understandable, but they remained unreliable. The model gave a reasonable account of one image containing people and shopping carts. In an empty crop, by contrast, it invented a woman, a man, and several shopping carts, and five of the eight responses simply hit the 128-token limit. That rules out this VLM as a counting tool or report source, although it works well enough as a clear demonstration of the HAT’s generative capabilities. A practical hybrid nevertheless begins to emerge. YOLO supplies dependable boxes and counting aids, an English-prompted VLM can describe selected crops when needed, and the 7B model turns the resulting facts into German prose, with everything checked deterministically.

Can the HAT and CPU work in parallel? #

The setup’s real strength appears when both processors run in parallel. I benchmarked the HAT YOLO pipeline and the 7B model on the CPU first on their own and then simultaneously. Both ran without memory pressure or application errors. The 16 GB is ample, and the 7B process continued to use about 8.5 GiB. The workloads are not completely independent, however. YOLO fell from 20.590 to 14.431 FPS, while 7B output dropped from 2.713 to 1.516 tokens per second, decreases of roughly 30 and 44 percent respectively.

The details explain why. Hailo inference itself slowed by only around 10 percent, while JPEG decoding dropped by 82 percent. The contention is therefore not mainly on the accelerator. It is for CPU time and memory bandwidth during decoding and preprocessing. Thermals remained harmless during combined operation, peaking at 68.3 degrees Celsius without thermal throttling.

The practical message is still positive. Even under load, YOLO runs at two to three times the planned 5 FPS. In a production system, I would nevertheless schedule large report jobs for another time, give them a lower priority, or limit them to fewer CPU cores so that the vision pipeline keeps its headroom.

Outlook: from parking lot to factory floor #

My parking lot was ultimately a proxy scenario, which is exactly why it is worth looking beyond it. Outside my test environment, I can see real value for a setup like this on a factory floor. Local detection and tracking can reveal how people move through a space, make worker routes and material flows visible, and help optimize routes and processes. Tracks can be turned into dwell times and movement patterns that reveal where paths continually cross, where bottlenecks slow work down, or where material travels farther than necessary. The same metrics can compare a layout before and after a change. The building blocks from this test fit the job surprisingly well. Analysis stays entirely on site, no image leaves the device, the privacy stage pixelates people before images are displayed anywhere, and the system analyzes movement patterns rather than identities.

Two limitations from the test apply there as well. First, tracking remains an approximation. Reliable process metrics require precise definitions and scene logic. Second, the shopping cart returns in another form. Forklifts, pallet trucks, and specialized load carriers are no more likely to be COCO classes, so a custom-trained model is unavoidable. In Germany, analyzing employee movement is also not merely a technical issue. A project like this cannot start without involving the works council and data-protection specialists early.

Conclusion #

The AI HAT+ 2 turns the Raspberry Pi 5 into a remarkably solid platform for continuous local image analysis. In object detection, the accelerator is 37.76 times faster than the CPU at inference and 19.64 times faster end to end. It sustains that performance for 30 minutes at maximum load without a single error and stays below 62 degrees Celsius. The division of labor is at least as valuable as the throughput. The HAT handles continuous computer vision, leaving the CPU available for I/O, tracking, privacy processing, and application logic, while 16 GB of RAM expands the system with a 7B language model capable of writing locally verified reports.

The HAT is not a universal AI accelerator. The small 1.5B LLM ran faster on the CPU, the VLM hallucinated, and no amount of acceleration can create an object class that a model does not know. In practice, success depends on the camera, the choice of model, and a pipeline that protects originals, treats privacy as a separate processing stage, and checks generated text against the facts automatically.

Test system #