Google Cloud announced the general availability of Managed Lustre, a high-performance storage solution powered by DDN's EXAScaler, offering throughput from 125 MB/s to 1000 MB/s per TiB and scaling up…
The llm-d project introduces co-operative time-slicing to interleave independent reinforcement learning jobs on shared hardware, increasing aggregate accelerator duty cycles from ~40% to 70% without i…
A new open-source tool called llm-d acts as an LLM-aware load balancer for distributed inference, intelligently routing requests across vLLM instances based on KV cache locality and GPU utilization. B…
NVIDIA open-sourced Grove, a Kubernetes API for managing multi-component AI inference stacks, at KubeCon Europe 2026. Grove introduces custom resources for gang scheduling, topology-aware placement, a…
IBM Research, Red Hat, and NxtGen Cloud Technologies demonstrated that using llm-d to serve AI models on mixed GPU hardware can boost inference speeds by 3 to 5 times and double throughput, enabling e…