Llama.cpp's default layer-split mode gives a dual-GPU setup the combined VRAM of both cards but runs single-stream generation at roughly the speed of one card, while tensor parallel can nearly double …
DecagonAI achieved a 4.7× improvement in fleet-level GPU efficiency on one of its largest scheduled inference workloads by pairing higher-memory GPUs with three serving changes: disaggregating prompt …
Cornelis Networks introduced its Active Compute Fabric (ACF) at the AI Infra Summit on Monday, an open architecture for scale-up and scale-out networking that integrates programmable compute into the …
Astera Labs, a Nasdaq-listed provider of rack-scale AI infrastructure, is hiring a Senior Software Engineer for AI for RTL Design & Formal Verification in San Jose, California, offering $161,000–$200,…
Nvidia is winning the AI race by focusing on data center interconnect and networking technologies rather than just raw chip performance, according to an analysis. The company's NVLink, Mellanox-based …
NVIDIA presented its Spectrum-X Ethernet Multiplane Network Architecture at Hot Chips 2026, claiming 1.9x higher training performance in multi-tenant AI factories and detailing a roadmap to scale AI f…
Nvidia is investing $6 billion to build a comprehensive AI infrastructure moat, focusing on interconnect technology, software abstraction, and energy-efficient compute to dominate the entire AI workfl…
NVIDIA introduced NVLink Fusion, a technology that connects custom XPUs to NVIDIA's AI infrastructure, aiming to reduce cost and time-to-market for AI factory builders. The solution offers 3x lower la…
Nvidia reported fiscal Q1 2027 data center revenue of $75.2 billion, up 92% year-over-year, accounting for 92% of total revenue of $81.6 billion. CEO Jensen Huang cited agentic AI as a key demand driv…
Cisco is championing an 'Ethernet-first' strategy for AI infrastructure, arguing that Ethernet is becoming the predominant networking technology for AI supercomputing due to its open standards, scalab…
A new arXiv paper (arXiv:2607.28633v1) proposes a topology-aware transfer orchestrator for disaggregated GPU inference, claiming existing systems like DistServe, Splitwise, and Mooncake ignore that ba…
NVIDIA CTO Michael Kagan said at WeAreDevelopers World Congress in Berlin that modern AI is no longer just about chips but about software, networking, and power, describing AI infrastructure as an "AI…
Data center switch spending for AI back-end networks is projected to reach $1 trillion by 2030, with scale-up interconnectivity accounting for more than half the total market, according to a report fr…
Cisco's new white paper, 'A Day in the Life of a Prompt,' warns that AI inference is becoming a networking issue as data movement emerges as the primary bottleneck for GPU performance. The paper forec…
Nvidia's NVLink bridge boosts prompt processing by 30% and FSDP training by nearly 3x on dual RTX 3090s, but token generation and DDP training show no benefit, according to tests by a developer runnin…
NVIDIA GB300 NVL72 set a world record for pre-training DeepSeek-V3 671B at 1,648 TFLOPs per GPU, demonstrating how advances across the AI platform—from silicon to networking to software—push training …
NVIDIA's NVLink and NVSwitch technologies form a scale-up fabric that connects GPUs tightly enough to behave as a single machine, contrasting with scale-out fabrics like InfiniBand and RoCE that link …
NVIDIA NVLink, the sixth-generation scale-up networking fabric for AI factories, delivers the highest GPU-to-GPU bandwidth and lowest latency all-to-all topology to enable accelerators to work as a si…
Nvidia Corp. networking chief Gilad Shainer argues that agentic inference transforms the network into part of the computer, making it a first-class participant in the computational process. In a speci…
Nvidia Corp. networking chief Gilad Shainer argues that in the era of agentic AI, the network has become a first-class participant in computation, transforming AI infrastructure from a server farm int…