Nvidia tackles AI factory scaling bottlenecks with open rack designs and networking advances
Nvidia Corp. today announced new partnerships and contributions to the open-source community at the Open Compute Project Global Summit in San Jose, aimed at laying the foundation for the future of agentic artificial intelligence infrastructure.
At the core of these announcements, the chip and hardware giant contributed key Vera Rubin rack and compute tray designs to the Open Compute Project through the company’s Nvidia MGX architecture. This will give system builders common blueprints for AI factories to develop future rack layouts, networking design, power and cooling systems.
“The real bottleneck for the industry right now is how fast the world can manufacture and deploy AI factories,” AI Infrastructure Solutions Lead Joe DeLaere said.
Through MGX 1.2, Nvidia is also providing open components that let engineers quickly scale and bind-mate liquid cooling and interoperable, rack-level interconnect infrastructure for high-power AI systems. It also adds modularity for liquid-cooled fanless I/O bays and standardized power, making it possible to streamline manufacturing and field installation.
The racks feature the Vera central processing unit rack design and extend the Nvidia MGX architecture beyond accelerated computing to handle the most intense agentic AI tasks.
Although the industry focuses heavily on graphics processing units, the workhorse of AI training and inference, when AI is deployed at scale, often CPUs get second-class citizen status. They are the orchestrators that do most of the behind-the-scenes work, dividing the networking, getting the data moved, sandboxing execution and crunching bits, while the GPUs do the parallel execution for the AI work.
“Between model calls, agents need to do orchestration,” DeLaera said. “They need to execute code. They need to process data, run evaluations.”
Without GPUs handling data-flow logistics and coordinating the application layer, the AI system quickly falls apart, like an army without a well-oiled supply line.
Nvidia claims that its Vera CPUs deliver around twice the performance of its previous Grace GPU generation on several workload benchmarks, and a 1.8-times improvement on Terminal-Bench, which measures practical agent tasks involving code containers.
Partnerships forge the way for the rest of the AI factory
As AI factories become more complex, the computing hardware is only part of the equation. Nvidia also announced that it partnered with Meta Platforms Inc., Google LLC and Cisco Systems Inc.
Meta is adopting Nvidia Spectrum-6 Ethernet switch for its next-generation AI factory network, which will connect directly to Vera-Rubin deployments. Spectrum-6 is Nvidia’s 102.4 terabit-per-second Ethernet switching chip designed to handle massive traffic and fabric networks, such as those generated when AI computing systems distribute data for work across thousands of processors or GPUs for training and inference.
Nvidia is adding support for Google’s Falcon Transport to its Spectrum-X Ethernet networking ecosystem, beginning with the company’s future ConnectX-10 SuperNICs. The roadmap also extends to BlueField hardware. Falcon is a transport technology designed to manage data delivery efficiently in large-scale networks. It can move traffic over multiple paths, respond to congestion, recover lost packets and maintain secure connections.
Also leaning into networking, Nvidia and Cisco are jointly defining a new optical module format: an Ultra-dense Small Form Factor Pluggable, or USFP, designed to support future 400-gigabit-per-second-per-lane networking fabrics.
This pluggable optical module acts as a removable component, allowing networking equipment to transmit and receive information over fiber-optic connections. The USFP design will offer four times the bandwidth density of the existing 200G OSFP form factor; Nvidia is claiming a 1.3-times improvement in power efficiency.
Across these partnerships, Nvidia is seeking to build out not just the CPU-scale for AI factory blueprints but also become the backbone for networking – the fabric that moves data between agentic AI and the applications that deliver and consume the outputs. More networking capacity can fit into the same physical footprint at lower power, allowing data center operations to expand their networks without larger switching systems.
Image: Nvidia
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos , powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.