Edge AI Power 2: Power Insertion with ElmorLabs

ElmorLabs offers a cost-effective power measurement solution for Edge AI accelerators using PCIe power insertion modules, tested with M.2 adapters. The setup involves connecting the PMD2 between the PSU and motherboard, and the PMD PCIe adapter between the PCIe slot and the M.2 AI accelerator.

In the first article, we gave an overview of Hailo’s power measurement solution. Series : Edge AI Power Benchmarking Part 1: Hailo-8, the Reference Methodology https://mariobergeron.com/posts/edge-ai-power-p01-hailo8/ - Part 2: Power Insertion with ElmorLabs this post Part 3: Measuring Edge AI Power with INA228 https://mariobergeron.com/posts/edge-ai-power-p03-ina228/ Part 4: Measuring the Power Efficiency of Axelera Metis https://mariobergeron.com/posts/edge-ai-power-p04-axelera-metis/ Part 5: Measuring the Power Efficiency of DeepX M1 https://mariobergeron.com/posts/edge-ai-power-p05-deepx-m1/ Part 6: Measuring the Power Efficiency of MemryX MX3 https://mariobergeron.com/posts/edge-ai-power-p06-memryx-mx3 In this second article, we attempt to reproduce the power measurements with an alternate strategy. The motivation is to determine a reliable power measurement solution, that we can apply to M.2 AI accelerators from other vendors, which do not have on-board power measurement functionality. Power Measurement solutionsThere are several companies offering power measurement solutions, at different price ranges. Most use the same technique as Hailo : measuring the voltage/current drop across a shunt resistor. In order to add this circuit into existing systems, the solutions offer adapters with power insertion capabilities. QuarchThe Quarch solution is based on a Power Analysis Module PAM , which connects to various fixtures for PCIe and/or M.2. In addition to power measurement, this solution comes with additional features, such as power generation, and fault injection. All this comes at a premium: Power Analysis Module PAM https://quarch.com/products/power-analysis-module/ : ~ 1, 500-2, 500 pounds PCIe Slot M.2 Power Injection Fixture https://quarch.com/products/pcie-slot-to-m-2-power-injection-fixture/ : ~ 300-600 pounds This solution was out of my budget, but it was relevant to mention just the same. ElmorLabsElmorLabs provides a solution that is compatible with standard ATX power supplies. They provide power measurement boards that are placed in line with the ATX24, EPS, and PCIE power cables, and also provide PCIe power insertion modules to measure the power on PCIe slots. Although the ElmorLabs does not address M.2 slots, their PCIe power insertion modules can be used with PCIe to M.2 adapters for our use case. ElmorLabs PMD2 https://elmorlabs.com/product/elmorlabs-pmd2/ : $99 USD PMD PCIe 4.0 Power Measurement Adapter https://elmorlabs.com/product/pmd-pci-e-slot-power-measurement-adapter/ : $30 USD- PCIe to M.2 adapter generic This solution is available at a modest budget, so I gave it a try. Custom INA circuitCreating a custom circuit with a INA device, similar to Hailo, would seem to be the best solution. There are several vendors selling INA-based circuits, that are meant to be integrated into a larger system: Adafruit INA228 I2C Power Monitor https://learn.adafruit.com/adafruit-ina228-i2c-power-monitor - $14 USD Adafruit INA23x DC Current Voltage Power Monitor https://learn.adafruit.com/adafruit-ina237-dc-current-voltage-power-monitor/overview $ 10 USD These modules would be the most cost effective, but are only part of a solution, and would require additional investigation and planning. Assembling the SystemFor my first attempt, I went with the ElmorLabs solution. This involved several steps, which I have illustrated in the following figure: First, the ElmorLabs PMD2 needs to be connected in series between the PC’s PSU and motherboard. Second, the ElmorLabs PMD PCIe Power Measurement Adapter needs to be connected in series between the PC’s PCIe slot, and our M.2 AI accelerator module. Connecting the PMD2 to our ATX power supplyThis step was rather involved, and required cutting all the convenient tie wraps that kept the cabling inside my ATX PC tidy. The ElmorLabs PMD PCIe Power Measurement Adapter allows for power insertion from an external 12V source provided by standard PCIE cables from a ATX power supply . Each of the following voltage rails can be sourced from the parent SLOT, or the external EXT 12V supply: - 12V : sourced directly from the SLOT or EXT 12V - 3V3 / 3V3 AUX : sourced directly from the SLOT or via on-board 3.3V regulator 2A driven by the EXT 12V As we will see in the next sections, this particular on-board 3.3V regulator can sometimes be insufficient due to its approximate 2A limit not documented, this is my best guess . - Hailo-8 : 3.3V 2A was sufficient for operation - Higher-power M.2 modules: 3.3V 2A may be insufficient — I observed at least one module that failed to enumerate when fed from the PMD’s on-board 3.3V regulator The simpler PCIe to M.2 adapter modules that you will find, simply route the PCIe’s 3V3 rail to the M.2 socket. The more complex and expensive PCIe to M.2 adapter modules will re-generate the 3V3 rail from the main 12V. This one from JEYI generates a 3.3V supply capable of 6A Depending on which M.2 AI accelerator is being measured, two possible strategies are possible, both involving on-board 3.3V regulators. The strategy on the left, uses the ElmorLabs PMD’s on-board 3.3V 2A regulator to feed the M.2 slot’s 3.3V. This works with the low-power Hailo-8 module, but not with higher-power modules I tested. The strategy on the right, uses the JEYI’s on-board 3.3V 6A regulator to feed the M.2 slot’s 3.3V. In both cases, the power comes from the external 12V supply, originating from the PSU’s PCIE1 cable, and being measured by the PMD2. Since my goal is to measure the power on several AI accelerators, I chose to go with the PCIe to M.2 adapter with the 3.3V 6A regulator. Measuring Hailo-8 Power with mb-powermon.pyThe ElmorLabs PMD2 Python API is available on github: In order to perform our testing, I have added support for the PMD2 probe in my mb-powermon.py utility The first step is to clone the repo for my open-source power monitoring utility. bash hailo virtualenv $ git clone https://github.com/AlbertaBeef/mb-powermon hailo virtualenv $ cd mb-powermon Within the hailo docker container, install the “pyserial” python package. bash hailo virtualenv $ pip3 install pyserial Make certain you have permission to access the PMD2 which will appear as /dev/ttyACM0, when connected to the PC bash $ ls /dev/ttyACM0 -la crw-rw---- 1 root dialout 166, 0 May 3 19:30 /dev/ttyACM0$ sudo chmod 666 /dev/ttyACM0$ ls /dev/ttyACM0 -la crw-rw-rw- 1 root dialout 166, 0 May 3 19:30 /dev/ttyACM0 Next, we can launch this utility as follows: bash hailo virtualenv $ python3 mb-powermon.py --probe hailo,elmorlabs --csv mb-powermon-hailo-pmd2-resnet50-20260502.csv If we re-run the hailortcli utility in a separate console within the Hailo docker container: bash hailo virtualenv $ hailortcli benchmark resnet v1 50.hef Starting Measurements...Measuring FPS in HW-only modeNetwork resnet v1 50/resnet v1 50: 100% | 20577 | FPS: 1371.25 | ETA: 00:00:00Measuring FPS and Power on supported platforms in streaming mode HailoRT warning Using the overcurrent protection dvm for power measurement will disable the overcurrent protection.If only taking one measurement, the protection will resume automatically.If doing continuous measurement, to enable overcurrent protection again you have to stop the power measurement on this dvm.Network resnet v1 50/resnet v1 50: 100% | 20581 | FPS: 1371.53 | ETA: 00:00:00Measuring HW LatencyNetwork resnet v1 50/resnet v1 50: 100% | 4472 | HW Latency: 3.10 ms | ETA: 00:00:00=======Summary=======FPS hw only = 1371.27 streaming = 1371.54Latency hw = 3.09513 msDevice 0000:01:00.0: Power in streaming mode average = 0 W max = 0 W hailo virtualenv $ hailortcli benchmark resnet v1 50.hef Starting Measurements...Measuring FPS in HW-only modeNetwork resnet v1 50/resnet v1 50: 100% | 20577 | FPS: 1371.26 | ETA: 00:00:00Measuring FPS and Power on supported platforms in streaming mode HailoRT warning Using the overcurrent protection dvm for power measurement will disable the overcurrent protection.If only taking one measurement, the protection will resume automatically.If doing continuous measurement, to enable overcurrent protection again you have to stop the power measurement on this dvm.Network resnet v1 50/resnet v1 50: 100% | 20581 | FPS: 1371.53 | ETA: 00:00:00Measuring HW LatencyNetwork resnet v1 50/resnet v1 50: 100% | 4445 | HW Latency: 3.10 ms | ETA: 00:00:00=======Summary=======FPS hw only = 1371.27 streaming = 1371.54Latency hw = 3.09517 msDevice 0000:01:00.0: Power in streaming mode average = 4.31512 W max = 4.31512 W NOTE: You may notice that hailortcli is reporting zero or identical average and max values e.g., 0W, 4.31512 == 4.31512 . As explained in Part 1, this is a signature of total or partial contention when two applications read the Hailo power API simultaneously — which is exactly what’s happening here, since mb-powermon.py is also polling the API for comparison. I recommended against this in Part 1, and I’m doing it deliberately here only to overlay both measurements on the same plot. The Hailo values shown should be treated as approximate references, not ground truth. For a clean Hailo reading, run hailortcli alone, as in Part 1. While this is running, you will see something similar to the following video playing at 10x speed : In this video, I am benchmarking resnet50 twice on the same Hailo module. If we convert the output.csv file to a user friendly.html, we can plot power and temperature for both runs and overlay Hailo’s reported averages: bash hailo virtualenv $ python3 csv-to-html-plot.py --input mb-powermon-hailo-pmd2-resnet50-20260504.csv --output mb-powermon-hailo-pmd2-resnet50-20260504.html The first thing that jumps out with these results are that the PMD2 measurements are significantly smaller than the Hailo’s own measurements, especially the lower values idle power . The PMD2 system was designed to measure much higher power values. A PCIe cable typically can handle up to 27 A continuous on the 12V rail ~324 W per channel . In comparison, Hailo’s power range is only 4W ~1% of PMD2’s range . This is likely at or below the ADC’s noise floor, where we are hitting quantization steps. ConclusionIn this article, we have implemented an independent power measurement strategy that can be used with any vendor’s M.2 AI acceleration module. It also works with these vendor’s PCIe accelerator cards, however, I do not have any in my possession. This strategy is fairly involved, and requires a standard ATX power supply, which is not always available, especially for the more compact mini-PCs and NUCs that have become very popular. Additionally, we have observed that the range of values we are measuring are too small for the PMD2, which was designed to measure much higher power consumption. The PMD2’s PCIE 12 V channels go up to ~324 W, whereas our measurement range is ~4 W, so we’re effectively operating at ~1% of PMD2’s full scale. This is likely at or below the ADC’s noise floor, where we are hitting quantization steps. In the next article, I will implement a simpler strategy using the Adafruit INA based power measurement boards. Read more javascript:void 0