Microsoft is finally enabling Remote Direct Memory Access (RDMA) over the USB4 protocol on Windows 11, allowing multiple PCs to be daisy-chained and used together to process large workloads. This means that Windows 11 now has a mechanism to distribute a workload across multiple interconnected PCs, enabling faster processing. Essentially, AI inferencing at home is now limited by the number of PCs and their specifications, rather than a single PC. RDMA allows the workload to directly access system memory on another PC, all through USB4, which offers up to 80 Gbps bandwidth for fast data transfer.
Instead of using Ethernet over USB4 in a UDP packet configuration, Microsoft chose to implement RDMA over USB4 to significantly reduce data access latency. Microsoft interconnected two PCs to measure round-trip latency by message size, finding that RDMA over USB4 achieves latency between 28 and 50 microseconds. In contrast, Ethernet over USB4 using UDP results in latency between 116 and 194 microseconds, making the RDMA implementation roughly four times faster. This lower latency will greatly benefit systems running large workloads, especially in latency-sensitive tasks like AI inferencing. Below DGX Spark mini PCs are connected over Ethernet, not USB4, but it serves the purpose of illustration.