South Korean edge AI chip startup DeepX has taped out its second-generation AI accelerator with access to custom processor-in-memory (PIM) technology from Samsung. The DX-M2 is designed to bring generative AI to robots, drones, and other edge devices, enabling inference of models with up to 100 billion parameters at less than 5 W.
DeepX’s first-generation chip, the DX-M1, is now in mass production, targeting vision applications. The newly taped-out DX-M2 aims to make local AI a practical choice for devices with tight power and thermal budgets. The chip is designed to run models with 20 billion to 100 billion parameters (approximately 3 billion to 20 billion active parameters) at 10 to 30 tokens per second, depending on the model. This is achieved within the same power budget as the M1—just 5 W.
“The most important change [between M1 and M2] is the memory bandwidth,” DeepX founder and CEO Lokwon Kim told EE Times.
Memory bandwidth has tripled from 50 to 150 GB/s, but there are several other efficiencies. DeepX said its NPU has doubled its power efficiency thanks to design improvements, and moving to a 2-nm process node doubled it again. 80 TOPS can now be achieved.
View All “This is important because with 5 W power consumption we can go everywhere, drones or any other battery-powered device,” Kim said. “Our goal is to enable deployment of sophisticated AI in all devices.”
The M2 can handle large mixture-of-experts LLMs and supports new techniques, including speculative decoding and TurboQuant.
Processor-in-memory
Most types of AI inference operations are constrained by moving weights between memory and compute, especially GEMV (general matrix-vector multiplication), during which random memory accesses are required, slowing down computation. Memory makers have come up with PIM schemes in which memory-bound operations such as GEMV can be computed inside the memory chip. Doing operations within memory increases the effective memory bandwidth and reduces time-consuming memory access times.
PIM is set to become a JEDEC standard for LPDDR6. However, DeepX has been working with Samsung on integrating the DX-M2 with Samsung’s LPDDR5x-PIM.
“We have spent one and a half years on this technology, collaborating with Samsung Memory, as they already have their custom PIM for LPDDR5x,” Kim said. “We already received LPDDR5x PIM samples and we are testing the operations and performance in-house.”
DeepX’s simulations show that off GEMV operations from the DX-M2 into LPDDR5x-PIM will result in 6.7× the tokens per second for Llama3.1-8B versus the DX-M2 alone.
“DX-M2 with LPDDR5x-PIM is close to optimal,” Kim said. “But our next generation, the DX-M3, will support official LPDDR6 PIM technology; then we can get the full theoretical performance advancement, which is 5× to 8× higher performance.”
The only drawback, Kim explained, is that the extra compute doesn’t come for free—6.7× the performance needs 2.7× the power consumption overall. That said, DeepX thinks it can get a 2× performance uplift within the same power envelope as non-PIM operation.
“Substantial manual effort” is required in the compiler to support this mode of operation, Kim said, noting DeepX currently has a prototype compiler running, based on a simulation of the PIM-enabled LPDDR5x. “We are now making the hardware modules so that we can test that in the physical environment, so we couldn’t finish the compiler yet, but we are actively working on it.”
Key markets
The DX-M2 will target several key sectors. DeepX is working with Taiwanese OEMs to find the right sweet spot for AI PCs, including laptops and workstations. Robotics companies, currently prototyping on GPUs such as Nvidia Thor, want lower power and more compute on-device, and automotive companies are looking for a low-power chip that can run agents in-vehicle, Tim Park, director of strategic marketing at DeepX, told EE Times. For IoT devices, DeepX is already working with Samsung on a next-generation smart hub. The company is also joining the AWS IoT ecosystem so it can use Greengrass to support remote model updates.
Robotics customers typically begin development on GPUs but have to look for alternatives due to thermal and power consumption considerations, Park said. DeepX is working hard to enable transitions from the Nvidia Isaac robotics platform to its in-house equivalent, Newton.
“Robotics customers ask us about our hardware, but also about how they can migrate from Nvidia to DeepX,” he said. “This is the answer—customers change their APIs to call Newton instead of Isaac and then they can run the exact same pipeline on the DeepX system.”
DeepX has 17 compatible libraries up and running, Park said. The company has demonstrated M1-based robots using the same source code as an Nvidia Jetson-based robot, but calling the DeepX libraries.
“The actual outcome for the customer is they can enjoy 20 times longer battery life and [lower cost] when they switch from Nvidia to DeepX,” Kim said. “At the same time, the switching time can be under a second. They just need to change the API name from Isaac to Newton. That’s it.”
A full robotics stack for DeepX hardware is currently under development.
DeepX is engaging with robotics customers in the U.S., China, Vietnam, and Korea, while Hyundai Motor Group Robotics Lab is already using the DX-M1 in its robotics platform.
Meanwhile, the company is building out its ecosystem, adding Taiwanese OEMs, vertical champions in various markets and, lately, a collaboration with Intel.
DeepX is working with Intel to integrate OpenVINO with the DeepX SDK in order to tap into Intel’s x86 customer base. The DX-M1 can complement integrated NPUs in Intel Core Ultra CPUs and enable AI for other Intel edge CPUs that don’t have an NPU, off some or all of the AI workload.
The company recently met with AMD to discuss jointly developing a robotics platform based on DeepX accelerators combined with AMD’s Kria reference board, in particular for the Korean ecosystem, Kim said. This is part of DeepX’s drive to collaborate with other heterogeneous ecosystems, including Raspberry Pi and Baidu’s PaddlePaddle ecosystem.
Commercial traction
DeepX has more than 100 purchase orders for the DX-M1 so far, from all major global markets across all physical AI industries (including drones, robots, smart factory, surveillance camera, aerospace and defense, healthcare, and IoT), Kim said. The M1 is now in mass production.
The most active market for physical AI was previously China, Kim said, but Japan has emerged this year as a major market, with a strong willingness to pioneer humanoid robots.
“All the large Japanese companies are evaluating our chips, more than 10 leading companies in Japan right now, and their feedback is very encouraging,” he said. “At the same time, the U.S. market [is strong]; we have several big names there, and their feedback is really good.”
DeepX is also engaged in smart city projects in South Korea, Vietnam, and Singapore.
“Our long-term view is that AI compute will become more distributed as efficiency improves,” Park said. “The PC and smartphone eras showed how computing can spread into many products. AI will follow its own path, but local inference has clear value when an application needs immediate access to sensor data or operation without continuous connectivity.”
DeepX considers edge AI to be a substantial opportunity for its chips, Park said, especially at a time when the amount of power available for new data centers is limited.
Kim’s belief is that just as only a quarter of the CPU market is in the data center today, the AI processor market will also shift to the edge in the next five to 10 years.
At a time when many AI data center projects are delayed, the AI capacity issue can’t be solved with investment alone, he added. AI chips that consume just 5 W can enable migration of AI to the device.
“If we deploy models up to 100 billion parameters in devices, we can reduce AI data center traffic by more than 80%,” Kim said. “That means we could have five times larger infrastructure without adding any AI servers. This is our vision.”
On the roadmap for DeepX’s third generation is the DX-M3, coming in 2028. The M3 will be chiplet-based and will target 1 ePOPS (INT8) using LPDDR6-PIM. The main target markets for this are automotive OEMs and humanoid robotics companies, whose demands will require multi-chiplet designs.
In the meantime, samples of the M2 are expected in March 2027, with mass production targeted for fall 2027.
Also read:
[Micron Bringing LPDDR5X to Desktops, Laptops, Data Centers](https://www.eetimes.com/micron-bringing-lpddr5x-to-desktops-laptops-data-centers/)
[Micron Unveils Nvidia-Friendly SOCAMM](https://www.eetimes.com/micron-unveils-nvidia-friendly-socamm/)