# NVIDIA NVLink: The Scale-Up Network for AI Factories > Source: > Published: 2026-07-20 15:46:28+00:00 The demand for AI continues to accelerate. Workloads are getting larger, models are becoming more complex, and there is mounting pressure to deploy AI compute infrastructure faster than ever. [AI factories](https://blogs.nvidia.com/blog/ai-factories-the-new-infrastructure-of-intelligence/)—data center-scale systems that continuously convert data and energy into intelligence—are being deployed to meet this insatiable demand. This AI factory approach to the data center has fundamentally changed system design and operation. Peak accelerator FLOPS are no longer enough. Today’s AI workloads, including trillion+ parameter models, mixture-of-experts (MoE) architectures, long-context reasoning, and disaggregated serving, require many accelerators working together as a single unit of compute. Achieving this requires high-bandwidth, low-latency GPU-to-GPU communication, fast in-network compute for collectives, and software-aware scheduling. Given the sheer number of components, resiliency needs to be built into the entire data center. Keeping up with the pace of AI requires technology and a supply chain that can move at the speed of industry innovation. This is why scale-up networking has become one of the most important architectural decisions in the AI factory. The scale-up fabric is what enables accelerators to work as a single unit of compute, determining how effectively tokens move across experts, how quickly collective operations complete, and how much useful throughput the factory can deliver. It is a significant factor in how much risk operators take when deploying new platforms. [NVIDIA NVLink](https://www.nvidia.com/en-us/data-center/nvlink/) is the purpose-built scale-up networking fabric for AI factories. It’s designed to accelerate AI inference, training, and other parallel computing workloads that require large, fast GPU-to-GPU communications. The Sixth Generation NVLink interconnect with NVLink 6 Switch provides the highest GPU-to-GPU bandwidth at the lowest latency all-to-all topology for scale-up networking, as well as support for SHARP in-network compute for offloading collective operations. It also includes rack-level resiliency features designed for production AI factory uptime. Developed through extreme co-design, where the entire stack from chips to systems to fabric to software are designed and optimized together, NVLink is part of the NVIDIA annual AI infrastructure roadmap cadence. This mature, proven, and widely deployed technology forms the backbone of modern AI infrastructure. ## How scale-up networking determines AI factory economics Scale-out networks connect servers across the data center. Scale-up networks enable the GPUs inside the domain to behave as a single engine of compute. Both are essential, but they solve different problems. Scale-out fabrics such as [NVIDIA Quantum InfiniBand ](https://www.nvidia.com/en-us/networking/products/infiniband/quantum-x800/)and [NVIDIA Spectrum-X Ethernet](https://www.nvidia.com/en-us/networking/spectrumx/) enable large clusters spanning thousands to hundreds of thousands of GPUs. This is critical for large, data-parallel AI training workloads (as well as large-scale scientific simulations). Scale-up fabrics connect accelerators in a single domain with high bandwidth, predictable low latency and shared high-bandwidth memory (HBM). For modern AI training and inference, the scale-up domain is where most latency-sensitive communication patterns occur. As an example, let’s take a look at inference with MoE models. Efficient inference implementations use [expert parallelism](https://developer.nvidia.com/blog/scaling-large-moe-models-with-wide-expert-parallelism-on-nvl72-rack-scale-systems/) to distribute experts across GPUs, and they use large batch sizes to maximize factory throughput. This parallelizes the workload, but it creates intensive all-to-all communication between GPUs. Tokens must be dispatched to the selected experts, processed, gathered, reordered, and passed forward. All of the GPU-to-GPU communication must happen in parallel. If the experts sit behind a low-bandwidth or high-latency fabric, gains from expert parallelism can be erased by communication overhead. The same phenomenon happens during training of MoE models. The takeaway is that all-to-all bandwidth *and *latency are critical to AI factory performance. AI factories need purpose-built, scale-up networking that is co-designed with the rest of the system to deliver and maintain these capabilities under load, across all workloads. Scale-up networking increases ROI by optimizing the delivered tokens per watt, per dollar, and per factory square foot. It leads to shorter training runs, higher utilization, and lower cost-per-token for AI factories. NVIDIA has demonstrated the impact of high-bandwidth, low-latency scale-up networking in large-scale MoEs. For models such as DeepSeek-R1 and Qwen 235B, as well as a simulated 2T parameter LLM, NVLink delivers up to 2.3X the decode throughput compared to leading off-the-shell (OTS) Ethernet. ## Evaluating scale-up technologies Spec sheets often compare fabrics using simple bandwidth numbers: link rate, aggregate switch capacity, or headline bandwidth per device. Those numbers are useful, but they aren’t sufficient. Evaluating scale-up capabilities for today’s AI workloads requires taking a factory-level view. Delivered, full-system performance determines how many tokens can be processed and produced per unit time, power, and factory footprint. It depends on all-to-all fabric bandwidth, end-to-end latency (that is, latency from every GPU’s HBM memory, through the fabric, to every other GPU’s HBM memory), the in-network compute for reductions and other collectives. It also requires fully integrated software at every level that can optimize routing, expose collectives, balance link traffic, pipeline data transfers, and has full support for the libraries and frameworks used by today’s AI workloads. Delivered performance only matters to the extent that the factory is operational. The ability to run for extended periods without failure, continuously monitor system health, and support service and maintenance at the component level while the rest of the factory continues operations translates delivered performance to goodput — the capacity of a factory to produce over its full lifetime. Achieving optimal delivered performance and goodput across many workloads through a complex technology is an incredibly difficult challenge, addressed through years of learnings with widely deployed scale-up infrastructure and a robust supply chain. Using an unproven technology stack or a solution not purpose-built for scale-up networking risks sub-optimal factory performance, disruptive downtime events, and undependable supply. Together, these metrics drive the only result that matters: sustained, dependable ROI over the lifetime of the AI factory. **The Key Metrics for Scale-Up Networking in AI Factories** Delivered Performance | Factory Resiliency | Platform Maturity and Proven Supply Chain | | Delivered factory performance on today’s AI workloads starts with bandwidth and latency, but also includes end-to-end scale-up network performance, in-network compute for reductions and other collectives, and a mature, full-stack software implementation integrated through every part of the solution. | Translating delivered performance to goodput requires system-wide resiliency, including long up-time, continuous system health-monitoring and telemetry, and support for service and maintenance at the component level while the rest of the factory continues to run. | Scaling AI factories is an incredibly complex challenge with many dependencies. Operators want to minimize risk by leveraging a mature technology stack with a proven extended track record of large-scale deployments and realized ROI. | *Table 1. These three key metrics are critical for evaluating a scale-up networking solution.* ## NVLink: Performance, resiliency, maturity Now in its sixth generation, NVLink provides the delivered performance and resiliency needed to maximize factory goodput, and over ten years of scale-up technology investment and proven deployments including world-leading hyperscalers, Cloud Service Providers (CSPs), and supercomputing centers. ### World-leading performance With [Vera Rubin NVL72](https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/), sixth generation NVLink provides 3.6 TB/s per GPU of bidirectional GPU-to-GPU bandwidth and 260 TB/s of rack-level GPU bandwidth in a 72-GPU domain. The end-to-end latency for GPU-to-GPU transfers is 3X lower than alternative solutions based on off-the-shelf Ethernet, and the packet rate is 10X higher. 6th generation NVLink delivers lower latency and higher packet rates for GPU-to-GPU transfers compared to alternative solutions based on off-the-shelf Ethernet, as well as in-network compute capabilities for collective operations NVLink Switch trays and NVLink spine of 5,000 cables form a single all-to-all topology so any GPU can communicate with any other GPU with uniform latency and bandwidth. Each tray includes four NVLink 6 switch chips, 28.8 TB/s of total tray bandwidth, and 14.4 TFLOPS of FP8 in-network compute. In a single Vera Rubin NVL72 rack, NVLink 6 provides 260 TB/s of aggregate bandwidth and 130 TFLOPS of in-network compute for accelerating reductions and other collective operations (e.g. all-reduce, reduce, broadcast, etc). The NVLink technology roadmap includes support for scale-up domain sizes up to 1152 GPUs and connectivity through co-packaged optics. Software is a critical piece of the scale-up networking stack and includes NVIDIA Dynamo, NVIDIA TensorRT-LLM, NVIDIA Collective Communications Library (NCCL) and more (discussed in detail in the section on platform maturity), all built on top of NVIDIA [CUDA](https://developer.nvidia.com/cuda), the world-leading parallel computing platform, first released nearly 20 years ago The hardware and software are developed with [extreme co-design](https://developer.nvidia.com/blog/building-for-the-rising-complexity-of-agentic-systems-with-extreme-co-design/). This results in performance speedups that can’t be achieved through siloed stack elements. For today’s AI workloads, this translates directly into infrastructure economics. As an example, in the transition from NVIDIA Hopper to NVIDIA Blackwell, which included doubling the NVLink bandwidth, expanding the size of the NVLink scale-up domain to 72 from 8 GPUs, and incorporating Dynamo for disaggregated inference, NVIDIA achieved a 50X improvement in MoE inference performance per watt. The NVIDIA [Vera Rubin platform](https://www.nvidia.com/en-us/data-center/technologies/rubin/) further extends this by doubling both NVLink bandwidth and in-network compute. As model architectures evolve, the hardware fabric and software communication stack evolve together. This extreme co-design approach to development requires tight collaboration across the entire stack. It’s not feasible to replicate with a siloed approach to development, especially under hyperscale deployment pressure, but it’s the difference between just adding accelerators versus scaling to useful delivered performance. ### Built for speed *and *resiliency AI factories must run continuously. As racks grow denser and more valuable, serviceability and fault management become part of the performance equation. A fabric that delivers high bandwidth but requires disruptive maintenance can reduce effective capacity and revenue. NVLink 6 introduces management and resiliency features designed for AI factory operations. The features make rack maintenance easier, provide better insight into component performance and load balancing, and keep the factory running even if individual nodes require service or updates. The sixth generation NVLink Switch supports factory resiliency features to maximize uptime and goodput: - Control plane resilience - Support for operation with partially populated racks - Hot-swappable switch trays - Software-defined routing with management controller fallback - Dynamic traffic rerouting - In-service software updates - Fine-trained link telemetry for monitoring and fault attribution These capabilities aren’t secondary. In a production AI factory, a failed link, switch, tray, or management controller shouldn’t force an entire rack out of service. Operators need fault isolation, telemetry, and service procedures that match the economic value of the infrastructure. NVLink 6 brings scale-up networking into the same operational discipline expected from the rest of the AI factory stack. ### Mature technology on a fast cadence The strongest technology strategies combine maturity with pace. NVLink has both. NVLink is now in its sixth generation of purpose-built scale-up networking fabric. It has been production-deployed at scale for nearly a decade, with millions of NVIDIA chips deployed across NVLink-capable systems and an ecosystem of servers, racks, cables, switches, software, and operations tooling. The NVIDIA annual platform cadence delivers new hardware capabilities at the pace of AI innovation, enabling the industry to rapidly deploy new models and workflows to meet the world’s AI needs. In addition to hardware, NVIDIA and the community release software as a key component of the scale-up networking stack. These include: **Dynamo****:** Open source framework for scaling generative AI and reasoning models in multi-node GPU environments, with disaggregated serving , dynamic GPU allocation, KV-cache-aware routing, and NIXL-based data movement.**TensorRT-LLM****:** Open source NVIDIA library for high performance LLM inference on NVIDIA GPUs, using optimized kernels, compute/communication overlap, and lower-precision formats like NVFP4 and FP8 to boost throughput including for MoE models.**NIXL****:** Open source NVIDIA library for fast data transfers across GPU memory, CPU memory, NVMe, and remote storage helping move KV-cache and inference state efficiently in distributed systems.: Open NVIDIA library for high-speed GPU communication, with more than 10 years of open source innovations. Includes topology-aware collectives, AI framework integration, and SHARP support for in-network reductions and other collectives.**NCCL****Fabric and Memory Management**: Includes address space management APIs, extensible memory semantics, and built-in memory coherency for efficient HBM access. Software innovations continue long after hardware release, enabling X factor speedups throughout the life of the product. ## NVLink-C2C extends the fabric to CPUs AI factories need more than GPU-to-GPU bandwidth. They also need high-bandwidth, coherent CPU-GPU connectivity for orchestration, data movement, memory management, storage services, and agentic workloads that mix compute phases. [NVIDIA NVLink-C2C](https://www.nvidia.com/en-us/data-center/nvlink-c2c/) provides that path. With [Vera CPUs](https://www.nvidia.com/en-us/data-center/vera-cpu/) in the Vera Rubin NVL72 platform, NVLink-C2C delivers 1.8 TB/s of coherent bandwidth between CPUs and GPUs, 7x the bandwidth of PCIe Gen6. This enables high-speed data sharing and a unified coherent memory architecture across CPU and GPU memory. For infrastructure teams, that means fewer bottlenecks between control, memory, and compute. For software teams, it simplifies programming models and supports workloads such as KV-cache offload, multi-model execution, data processing, and agentic orchestration. Vera itself is designed for this role, with 88 NVIDIA custom Olympus cores, high memory bandwidth, and power-efficient operation for AI factory workloads. In the NVIDIA platform, CPU, GPU, NVLink, HBM, system memory, networking, and software are co-designed to optimize performance. ## NVLink Fusion: Semi-custom XPU infrastructure without starting over Hyperscalers and AI natives build specialized silicon for targeted workloads and to offer options to their customers, but they face several challenges in deploying them. Integrating state-of-the-art scale-up networking, designing and deploying a complete rack-scale architecture, committing to data center design amid silicon supply risk, and supporting heterogeneous infrastructure each present unique difficulties. [NVIDIA NVLink Fusion](https://www.nvidia.com/en-us/data-center/nvlink-fusion/) provides that path. NVLink Fusion is the high-bandwidth, low-latency interconnect technology and IP that connects custom silicon to the NVIDIA world-leading AI infrastructure platform. With NVLink Fusion, they can leverage the proven NVLink scale-up stack and ecosystem to reduce development and deployment complexity, increase performance, and accelerate time-to-market for semi-custom AI factories. And by standardizing on a single unified architecture, NVLink Fusion simplifies operations across the data center, enables flexible reprovisioning of data center capacity, and allows custom AI XPUs to integrate seamlessly with GPUs for disaggregated compute. This breaks a fundamental constraint by eliminating the need to choose between custom XPUs and a world-class AI platform. And because NVLink Fusion keeps pace with the NVIDIA technology roadmap, adopters can keep pace with the company’s annual cadence. ## The path for production AI factories The next AI factory won’t be won by the fastest standalone accelerator. It will be won by the infrastructure that can deliver the most intelligence, at the lowest cost per token, on the fastest cadence, with the least deployment risk. NVLink delivers leading performance, with 3X lower latency and 10X higher packet rates as well as 130 TFLOPs of in-network compute, factory resiliency features, and a mature, proven platform. AI factories are becoming the defining infrastructure of the AI computing era. NVLink delivers the performance, operability, maturity, and continuous innovation that powers them. Learn more about the [NVIDIA Vera Rubin Platform](https://www.nvidia.com/en-us/data-center/technologies/rubin/), [NVLink](https://www.nvidia.com/en-us/data-center/nvlink/), and [NVLink Fusion](https://www.nvidia.com/en-us/data-center/nvlink-fusion/).