Scality launched its AI Inference Factory, an on-premises inference stack that pools a shared KV cache across GPU servers and separates prefill from decode to improve GPU efficiency, CEO Jérôme Lecat …
CoreWeave Inc. launched CoreWeave Forge, a platform connecting serving, observability, post-training and evaluation, and previewed CoreWeave RL Rollouts, which improved model reload latency by 15x ver…
Scality launched Scality AI Inference Factory, an open-code software stack that serves the KV cache for open-weight model inference from its AI Data Infrastructure object storage over RDMA, which Scal…
Vast co-founder and CTO Alon Horev said a long-running AI agent session of half a million tokens can consume one-tenth to one-twentieth of a GPU's memory, and that Vast's tiered storage approach offlo…
Independent audits of real production traffic put physical KV cache hit rates in the 70s on average, while the logical hit rates reported in agent traces can run 90% to 99%, a gap of twenty to thirty …
Groq Inc., an AI startup, announced it has raised $350 million in a Series A round led by Disruptive, with Nvidia Corp. planning to join later. The funding comes less than three months after Groq clos…
NVIDIA Dynamo, an Apache-2.0 datacenter-scale inference orchestration layer from NVIDIA, sits above vLLM, SGLang, and TensorRT-LLM to coordinate multi-GPU clusters, with published benchmarks showing u…
UNIT AI, founded by Guy Glass, raised $12 million from Prologis Ventures and Dynamo to modularize warehouse automation. The funding will accelerate the company's development of flexible robotics syste…
NVIDIA claims its Blackwell NVL72 platform delivers up to 25x performance per watt over the Hopper generation for frontier AI models, positioning performance per watt as the critical metric for AI inf…
Nvidia is developing its proprietary CMX platform to improve AI inference economics by offloading KV cache to SSDs via BlueField DPUs, while the open CXL standard offers a vendor-agnostic alternative.…
DDN launched the A1400X3M storage appliance, delivering up to 160 million IOPS per rack, and introduced KV Cache software integrated with Nvidia Dynamo to accelerate AI inference. The new hardware and…
SGLang achieved a 5x throughput improvement for serving DeepSeek-V4 on NVIDIA GB300 hardware since its Day-0 launch, reaching ~11,200 tok/s/GPU at 50 tok/s/user through kernel and runtime optimization…
Fortinet announced an expanded integration of its FortiAIGate solution with Nvidia's AI platforms on May 12, embedding zero-trust security directly into Nvidia's Blackwell and Hopper GPUs to protect a…
IBM, Supermicro, and Nvidia engineers published an IBM Redbook detailing a reference architecture that uses IBM Storage Scale Erasure Coding Edition on Supermicro servers as an external shared storage…
NVIDIA has introduced Dynamo Snapshot, a checkpoint/restore approach for AI inference workloads on Kubernetes that reduces cold-start latency from minutes to near-instantaneous startup times. The solu…