17:08
2026-07-14
aleksagordic.com
artificial-intelligence
TPU and GPU Clusters: The Anatomy of Collective Communication
Google's TPU clusters use 2D or 3D torus topologies with 4 or 6 nearest neighbors per chip, while NVIDIA GPU clusters rely on fat-tree topologies over InfiniBand, according to a July 14, 2026 technica…