NCCL — Web Pulse coverage

AllReduce Stalls Are Network Stalls. Most Tools See Neither. :: https://wpnews.pro/news/allreduce-stalls-are-network-stalls-most-tools-see-neither
Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL :: https://wpnews.pro/news/shipping-a-trillion-parameters-with-a-hub-bucket-delta-weight-sync-in-trl