Mac Thunderbolt4 real Latency numbers

A Level1Techs forum user posting as boatman reported that USB RDMA on Macs is "just good software" and not real RDMA hardware offload, after being unable to obtain ib_read_lat, ib_write_lat or ib_send_lat latency numbers on a two-node Mac Thunderbolt setup. The user measured 7.8uS over TCP on Strix Halo with busy polling enabled via net.core.busy_poll=100 and net.core.busy_read=100, and set thresholds of greater than 5uS for software RDMA, under 2uS for genuine hardware RDMA.

Hello All, Not posted in a long time. but I cant find answers anywhere else. I have been following the AI clustering topics about USB4. I need someone to help get me real latency numbers for Mac Thunderbolt RDMA. Can someone run ib read lat, ib write lat and ib send lat or some equivalent on a 2 node setup. I need to know if it is real RDMA or just really good software RDMA. My understanding right now is that there is no way that thunderbolt can bypass the CPU/kernel and go to the GPU vRAM with a zero-copy. There is a limit to how low they can get without adding a RDMA chip to the USB port, which apple might do. If results are 5 than it is just software RDMA. If results are lower than they must have RDMA hardware. If it is <2uS then we are really cooking. Notes: I have tested both thunderbolt-ibverbs and OdinLink-Five . and they are good but most of the improvements they get in latency is from enabling busy polling. I am running it with Strix Halo and getting 7.8uS over TCP. I want to know if Apple is just doing the same trick. sudo sysctl -w net.core.busy poll=100 sudo sysctl -w net.core.busy read=100 I look forward to getting real numbers for latency. boatman https://forum.level1techs.com/u/boatman 2 Update: I was not able to get numbers but I can confirm that the USB RDMA on Macs is just good software. not real RDMA hardware offload.