TL;DR: At Autonomous Vehicles & AI USA 2026, Dr. Mohamed El-Hadedy showed that AV perception sharing a central computer with logging, OTA updates and other services can slow down about 15x while every health metric looks normal. His group’s runtime “perception integrity contract” checks frame rate, output stability and context freshness every five seconds, and held perception at 27–30 frames per second under stress on edge hardware.
Fixstars exhibited at Autonomous Vehicles & AI USA 2026 in Anaheim on August 26–27. Of the sessions we followed, the one closest to our day-to-day work was an invited talk by Dr. Mohamed El-Hadedy of the Reconfigurable Space Computing Lab at Cal Poly Pomona and the Coordinated Science Laboratory at the University of Illinois Urbana-Champaign: “Perception Integrity Contracts for Runtime-Trustworthy Autonomous Vehicles.”
He opened with three questions. Every AV safety case can answer “Was perception validated?” Very few can answer “Is perception OK right now?” And almost none can say whether they will be able to prove it in 2045, when a vehicle shipped today may still be on the road.
Key takeaways #
- On a shared central computer, perception can slow down by an order of magnitude while every platform health metric still reads normal.
- Watchdogs, resource monitoring and redundant pipelines don’t catch perception that is “slow but alive,” especially when redundant copies run on the same silicon.
- The proposed fix is a runtime contract that checks what perception actually delivers every five seconds and feeds failures back into scheduling.
- The supporting crypto and logging layers were measured on low-power edge hardware, at a cost of milliseconds and single-digit watts.
- All results so far are from bench and cluster-class hardware. Road and fleet validation is the next step.
A data center on wheels, and who loses when it gets busy #
The central computer in a modern AV runs perception alongside planning and control, data logging, OTA updates, infotainment and diagnostics. These services compete for processor time, memory bandwidth, data paths and thermal headroom. Logging tends to come in bursts, and OTA updates arrive without warning. When other workloads spike, perception is the one that degrades, and it does so quietly.
We heard a similar “data center on wheels” theme from Valeo at this year’s Embedded Vision Summit. Dr. El-Hadedy’s talk picks up where that architecture story leaves off: once everything shares one SoC, what guarantees that the safety-critical workload still gets what it needs?
One slide made the problem concrete. On the platform side, every service was running and CPU, memory and network all looked normal. Meanwhile perception was running about 15 times slower than intended, with flickering detections and stale location context. As he put it, “The platform measures effort — not eyesight.”
He then translated frame rates from his experiments into distance at highway speed. At 65 mph a car covers roughly 29 meters per second. At 30 frames per second, that is about one meter between looks at the road. At 2 frames per second it is around 15 meters, and at 0.8 it is roughly 36 meters.
Why existing safeguards miss it #
Each conventional safeguard covers a different failure mode. Hardware watchdogs catch software that has crashed or frozen, not software that is running slowly. Resource monitoring catches an overloaded computer, not degraded results on a computer that looks healthy. Design-time validation measures model quality before shipping, not what interference does to the model in the field. Redundant pipelines protect against a single hardware fault, but two copies on the same shared computer starve together.
Perception integrity contracts #
The core of the talk is a runtime monitor his group calls SEKHMET. In every five-second window, it checks three conditions:
- Timing: is perception producing frames fast enough, with bounded latency?
- Stability: do its outputs stay within an expected range of variation inside the window?
- Context: is each frame paired with sufficiently fresh GPS data?
Each window gets a pass or fail, and silence counts as a failure. The key design choice is what happens next. A failed window isn’t just written to a log. It goes to whatever makes scheduling decisions, which can defer an OTA update, logging, evict lower-priority workloads, or move the vehicle toward a safe mode if failures continue.
The measurements were done on a Raspberry Pi 5 with a Hailo-8L accelerator running YOLOv8s on unmodified K3s. Without the contract, perception dropped to roughly 0.8–2.2 frames per second under stress, while the pods kept reporting Ready. With it, perception held 27–30 frames per second across 400 stressed windows.
Each verdict is also stored as a 96-byte receipt that includes the hash of the previous one. If anyone edits the record later, the chain breaks, and an outside party such as a regulator or investigator can detect it.
The rest of the stack #
The talk covered several other layers, each measured on its own:
- Sealed sensor input (HOURS): each telemetry message is authenticated with ASCON-128a, the NIST lightweight cryptography standard, before the pipeline sees it. In a live run of 7,020 messages, 99.8% verified, with about 25 ms mean latency at roughly 200 packets per second and about 30% less energy per packet than AES-GCM on a 7 W cluster.
- Per-inference gating (OSIRIS): an inference result is acted on only if its confidence clears a threshold and it arrives before a deadline. Otherwise the system abstains or denies. The end-to-end budget is 100 ms, with the accelerator path at 40 ms or less on ARM and RISC-V edge boards.
- Crypto in reconfigurable hardware (MEHEN): about 0.46 ms per crypto call on a vehicle-class FPGA at 327 MHz, with the board drawing 4.21 W.
- Post-quantum sealing (KHEPRI / KHONSU): receipts are signed with LMS hash-based signatures at a median of about 20 ms. Using Mosca’s rule, he argued that evidence from a vehicle shipped in 2026 has to stay trustworthy past 2045, so migration planning should start from the fleet’s service life rather than from a guess about when quantum computers arrive.
He was also clear about what he is not claiming yet. The numbers come from bench and cluster-class hardware, not vehicles on public roads. The layers were measured individually, not as one integrated stack. And the fleet-wide version, where every car acts as a node, is still a research direction. His request to the audience was a pilot: ten vehicles for six months.
The Fixstars perspective #
For a team that spends most of its time on performance engineering for embedded AI, the “green dashboard” slide looked very familiar. Average utilization and a “running” status say little about whether a latency-sensitive workload is meeting its deadline. The real problems tend to show up in tail latency and in interference from things that were never part of the benchmark: a logging burst, a background update, a thermal limit. A few practical points we took away:
- Define perception health by its output: frames per second, P95 latency and input freshness. CPU percentage is not a substitute.
- Benchmark under realistic co-located load, not in isolation. Results with logging, OTA and other services running are the ones that matter.
- Treat compute headroom as a safety resource. Time saved through model optimization, quantization or better scheduling is margin that can absorb interference or pay for checks like inference gating and message authentication.
- Map runtime contracts onto mechanisms the platform already has. His adoption slide listed AUTOSAR Adaptive state management, hypervisor partitioning, cgroups and core reservation as in-vehicle equivalents of the Kubernetes controls used in the prototype.
Conclusion #
The main message of the talk is that runtime evidence should be an engineering requirement for autonomous systems, alongside design-time validation. The receipts and post-quantum signatures are interesting in their own right, but the more immediate lesson for embedded teams is simpler: measure perception by what it delivers, under the load it will actually face.
FAQ #
What is a perception integrity contract?
A runtime check, proposed by Dr. Mohamed El-Hadedy’s group, that evaluates an autonomous vehicle’s perception output in fixed time windows (five seconds in his experiments) against three conditions: frame rate and latency, stability of model outputs, and freshness of context such as GPS. A failed window goes to the scheduler, which can defer updates, logging or move the vehicle toward a safe mode.
Why don’t watchdogs and resource monitoring catch degraded perception?
Watchdogs detect software that has crashed or frozen, and resource monitors detect overloaded hardware. Perception that is still running but slowed by interference from other workloads on the same computer passes both checks. In the experiments presented, perception slowed about 15x while all platform metrics read normal.
How much do the extra security and logging layers cost on edge hardware?
In the bench measurements presented, each verdict record took 96 bytes, message authentication with ASCON-128a averaged about 25 ms under load, hardware crypto calls took about 0.46 ms on an FPGA board drawing 4.21 W, and post-quantum LMS signatures took about 20 ms per record.
What does this mean for embedded AI optimization?
Perception health should be measured by its output (frame rate, tail latency and input freshness) under realistic co-located load. Optimization that cuts inference time adds headroom that can absorb interference from other workloads or pay for runtime checks.
If you’re working to get AI models running reliably within the compute, power and latency limits of an in-vehicle or edge platform, our AI Model Porting & Optimization team would be glad to talk. Moving AI onto embedded or edge hardware?
Fixstars engineers port and optimize AI models and embedded software for your target chips, including NVIDIA Jetson and DRIVE, Renesas R-Car, and Qualcomm Snapdragon.
Author
Miho Yoneda
Marketing Lead at Fixstars Solutions