{"slug": "google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets", "title": "Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets", "summary": "Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed large language model inference. Version 1.0 adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs.", "body_md": "Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference. It adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs.\n\nThe post [Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets](https://www.marktechpost.com/2026/10/20/llm-d-1-0-open-source-distributed-inference-stack/) appeared first on [MarkTechPost](https://www.marktechpost.com).", "url": "https://wpnews.pro/news/google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets", "canonical_source": "https://www.marktechpost.com/2026/10/20/llm-d-1-0-open-source-distributed-inference-stack/", "published_at": "2026-10-21 04:33:40+00:00", "updated_at": "2026-10-21 07:13:41.574303+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools", "mlops", "developer-tools"], "entities": ["Google", "Google Cloud", "Red Hat", "IBM", "llm-d", "vLLM", "SGLang", "Kubernetes"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets", "markdown": "https://wpnews.pro/news/google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets.md", "text": "https://wpnews.pro/news/google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets.txt", "jsonld": "https://wpnews.pro/news/google-cloud-ships-llm-d-1-0-with-cache-aware-routing-for-tpu-and-gpu-fleets.jsonld"}}