LeaderWorkerSet

Google benchmarked a layered routing architecture for its multi-cluster GKE Inference Gateway across a multi-region deployment of 17,000 compute nodes in us-east5, us-west8, and europe-west4, serving …

Modelplane released version 0.4, adding an NVIDIA Dynamo serving stack that brings gang scheduling and peer-to-peer weight transfer to its inference clusters. The release, built with NVIDIA's Dynamo t…