A Stanford professor published a talk this week claiming TCP is the wrong protocol for AI clusters — and the numbers back him up. John Ousterhout, professor emeritus at Stanford and creator of the Tcl language, presented “Homa: The End of TCP for AI Clusters” at the AI Engineer Summit, which went viral on Hacker News this week. His Homa protocol cuts short-message P99 latency from 1.2 milliseconds to 92 microseconds — a 13x improvement — by solving a structural problem TCP was never designed to fix.
Why TCP Fails AI Inference #
TCP was designed to move bulk data reliably across unreliable networks. That design decision — treating traffic as an undifferentiated byte stream — turns into a liability inside a GPU cluster. When AI nodes finish a compute phase, they exchange small coordination messages before the next phase begins. Those messages queue behind whatever large data transfers are already in flight. Short messages wait. GPUs wait. Expensive compute time burns.
This is head-of-line blocking, and it is fundamental to how TCP works — not a tuning issue. Compounding the problem, TCP uses sender-driven congestion control, which reacts slowly to the incast patterns common in distributed inference, where many nodes simultaneously send traffic to a single receiver. As a result, the sender oscillates between over-sending and under-sending, keeping tail latency high. As Ousterhout puts it: “Even a millisecond of latency causes an expensive GPU to go idle.”
At current GPU pricing, that is not metaphorical. Indeed, a cluster of A100s costs thousands of dollars per hour. Consequently, every synchronization barrier is a mandatory — and TCP makes that longer than it needs to be.
How Homa Fixes the TCP AI Cluster Problem #
Homa solves the problem through three coordinated mechanisms. First, messages in Homa have explicit sizes — the receiver knows exactly how large an incoming message is from the first packet. This enables Shortest Remaining Processing Time (SRPT) scheduling: short messages skip ahead of long ones in the queue, the same way a cashier waves through a customer buying one item when there is a full cart ahead.
Second, the receiver controls congestion, not the sender. Instead of reacting to delayed ECN signals, the receiving endpoint grants explicit transmission permission based on current network state. No oscillation, no incast collapse. Third, Homa exploits switch priority queues to enforce this ordering at the hardware level — short, latency-sensitive messages get physical priority on the wire.
The results are concrete. On a 100 Gbps network at 80% utilization, Homa achieves 92 microseconds P99 latency for short messages versus 1.2 milliseconds for TCP — 13x lower. Moreover, for longer messages, performance improves by approximately 2x. A less obvious finding: running Homa alongside TCP also improves the performance of the remaining TCP traffic.
Related: Volantis Raises $88M to Give GPUs 220 Memory Chips Instead of 8
Homa Linux Kernel Module: What Is Available Now #
Homa is not vaporware. The PlatformLab/HomaModule repository on GitHub provides a Linux kernel module you can install without a system reboot. In March 2026, Homa was backported to Red Hat Enterprise Linux 8 and 9.5. Additionally, a new homa_qdisc queuing discipline, added in January 2026, lets TCP and Homa run simultaneously on the same host — enabling incremental migration rather than a hard cutover. Meanwhile, Ousterhout is actively drafting an IETF standardization document and pursuing Linux kernel upstreaming, which began in October 2024. Financial services firms are reportedly already running prototype implementations.
Why Adoption Will Not Happen Overnight #
Networking engineers are skeptical, and they have precedent on their side. DCCP was a previous attempt to improve on TCP for specific workloads. It failed because the internet runs on TCP — firewalls, NAT devices, and routers pass TCP traffic and drop everything else. As one engineer in the Hacker News discussion noted: “I’ve heard about half a dozen protocols that will replace TCP over 20 years — and yet here we are.” Network architect Ivan Pepelnjak called Homa “a solution looking for a problem” in a 2023 critique, though that assessment predates the current inference surge.
However, Ousterhout is explicit that Homa targets data centers where you control the full stack — kernel, switches, and application layer. The middlebox problem does not apply inside your own cluster. Furthermore, the harder barrier is application-level changes: Homa is not backward-compatible with TCP, so existing applications need to be ported. That is not a trivial ask, even for organizations that want the performance gains. Nevertheless, if you control the infrastructure, the case is clear.
Key Takeaways #
- TCP’s head-of-line blocking is a structural problem for AI inference clusters — short coordination messages queue behind large transfers, idling expensive GPUs during every synchronization barrier
- Homa cuts short-message P99 latency 13x versus TCP (92µs vs 1.2ms) using receiver-driven congestion control and SRPT message scheduling on a 100 Gbps network at 80% utilization
- Homa is available today as a Linux kernel module with RHEL 8/9.5 backport; the
homa_qdiscdiscipline supports concurrent TCP+Homa operation for incremental migration - Adoption requires application-level changes and switch reconfiguration — this is not a drop-in replacement, and networking skeptics citing DCCP’s failure are right to be cautious
- If you run AI inference clusters at scale, profile your short-message traffic share and evaluate Homa seriously — the GPU compute cost of TCP latency is real and measurable