SANTA CLARA, Calif. — Ambarella is using AI agents to abstract away its software development kit, helping new independent software vendors (ISVs) port their applications to Ambarella hardware in a matter of days instead of the months it previously took, Muneyb Minhazuddin, customer growth officer at Ambarella, told EE Times.
“The traditional way for [chip companies] to reach developers has been to make APIs available, publish on GitHub, build communities, do training, and have a large amount of folks coming in and developing code on your system,” Minhazuddin said. “That’s a lot of resources, a lot of people, a lot of money, and a lot of time.”
Ambarella has put an agentic layer around its APIs and SDKs that developers can interface with. This agentic layer has enabled ISVs to port code to Ambarella hardware in a matter of days, versus the months they would have spent previously, Minhazuddin said.
“[At first, we thought] it must be down to the ISV’s smart engineers, or the technology, or our products,” he said. “But using the same formula with subsequent ISVs, it’s the same thing. It’s happened half a dozen times. This actually works.”
View All Compared to the majority of data center-scale applications, embedded software is considerably simpler and doesn’t require huge models to analyze.
“The good news is, at the edge, it’s very fixed-function,” Minhazuddin said. “It’s not that broad, so we can take these very fixed functions and turn them into agent skills, and then express them through an agentic layer to orchestration engines to build applications.”
Embedded software could be one of the first sectors to fully adopt agentic AI for development, Minhazuddin said.
“A lot of issues with MCP [model context protocol] and agentic AI are still true in the large data center and cloud models because the number of permutations is so high,” he said. “What we’re finding is [that the problem] is more confined at the edge, so it’s actually more effective for us.”
Ambarella has onboarded ISVs onto its hardware platform across healthcare, retail, and robotics so far, some in as little as three days using the agent, Minhazuddin said, but the real test is whether they can translate this speed into faster proof-of-concept systems for end customers.
“In the last four months, [our ISV partners] have put our development kits into 20 large retail chains in the United States, large chains of coffee shops, and drive-throughs,” he said.
On the retail side, many customers already have some kind of networking box on the premises, with a second box that serves point-of-sale systems and other applications that might run in the store. A third box can be added for AI inference on camera feeds and other analytics.
“What’s emerging as an interesting pattern is: Can you consolidate those three boxes?” Minhazuddin said. “Can you have a network function, a point of sale, an application system, and an AI function [in the same box]. There is a significant reduction in capex that you can achieve if you’re able to consolidate three different workloads onto one system.”
Retail industry customers are evaluating Ambarella SoCs for security camera aggregation and analysis as well as other AI functions, Minhazuddin said, noting that combining several boxes at several thousand dollars each could make a big saving across a chain of tens of thousands of stores.
Agents at the edge
Agents will also need to run directly on edge devices eventually. For most other types of AI applications, a system’s CPU runs as a host, and most of the compute is done on a GPU or accelerator. With agentic AI, the CPU is needed to run an agentic engine.
“The CPU-GPU ratio is changing fast because of agentic AI,” Minhazuddin said. “We already have an Arm CPU core in our SoC… we can balance the CPU and on-chip accelerator to run agentic functions quite naturally.”
Is Ambarella looking at changing the ratio of CPU to acceleration on its hardware to suit agentic applications?
“Maybe in future products,” Minhazuddin said. “That would align with where the industry is going.”
Agents could run in an edge box or in the camera, or both, Minhazuddin said. Ambarella’s edge box design features a single N1-655 chip; the company has publicly demonstrated inference for a 20B MoE model and now has 35B models up and running.
“[An N1-655] is good enough to do local processing, inference, agentic, all of that, to a well-controlled degree,” he said. “Camera-box combinations can eliminate requirements for big servers.”
How to distribute workloads between the camera and the edge box is a customer decision, Minhazuddin said.
“We’ll continue to add more oomph into the camera itself,” he said. “If you can aggregate those different streams, then you don’t have to have a [rack-scale] server; you can still be a small appliance; that’s the pattern.”
Edge to cloud
Minhazuddin said the edge AI market is evolving in two directions: cloud to edge and edge to cloud.
In the cloud-to-edge sector, data center owners want to move inference to the edge using multi-rack-scale systems as an on-prem extension of their infrastructure, still with huge model sizes and powerful compute hardware. The software ecosystem here is similar to enterprise software, Minhazuddin said.
The edge-to-cloud sector has two sub-sectors, with industry-specific software ecosystems, and Ambarella is addressing both of them, he said.
One is physical AI. In previous iterations of the IoT, sensor data was digitized and analyzed, but humans made the decisions. Modern physical AI applications aggregate data from multi-modal sensors and combine them to faithfully represent a physical system with a higher level of autonomy, Minhazuddin said.
“Higher autonomy means these systems need to not just perceive and sense; you have to make some decisions locally, and once you make a decision, you have to take action, so it’s fully self-sufficient,” he said. “If this system was tethered to the cloud, the decision to move a robotic arm would have to go to the cloud… it has to be fully autonomous.”
The other edge-to-cloud sector is the edge box, sufficient to aggregate data from multiple sensors collecting data elsewhere, perhaps combining multiple sensor modalities, perhaps with multiple models running.
“The aggregation point doesn’t have to be the cloud, and you don’t really need a massive server,” he said.
The N1-655 is being positioned in both these edge-to-cloud segments, he said, but multi-chip Ambarella systems may have to wait a bit longer.
“Architecturally, we have a lot of work to do to get there,” he said. “Our performance per watt is highly efficient on our SoC, but if we start adding a lot of SoCs, then we need HBM, PCIe… Could you take a lot of these SoCs and put them on M.2 cards and connect them, yes, but would it be a truly multi-chip cluster architecture? Not really, no.”
AI workloads will probably continue to swing back and forth between the edge and the cloud, Minhazuddin said, noting that AI began in the data center, moved closer to the cloud, and is emerging at the edge.
“At some point, there’ll be some level of equilibrium where some workloads run best in the cloud, some in the data center, and some work best in the edge, but they’re all talking to each other; they’re all connected to each other,” he said. “That’s where we’ll hit the equilibrium, but I don’t think we’re there yet.”
Read also:
SiMa Launches Agentic Development Environmen****t for Physical AI
Edge AI chip company SiMa.ai has launched an agentic development environment designed for developers of physical AI systems working with the company’s Modalix MLSoC chip.
**Scaling Down Is the New Scaling **Up
Agentic AI on edge devices will require hardware-aware design from the start, according to Meta’s Vikas Chandra, a keynote speaker at the Embedded Vision Summit.