Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference. It adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs.
The post Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets appeared first on MarkTechPost.
source & further reading
marktechpost.com — original article
Mistral AI Releases Voxtral 3: Speech-to-Text, TTS and a Voice Agent SDK
DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash
Exa 3.0 Search API Tops Perplexity, Brave and Parallel on 8 Benchmarks