{"slug": "nvlink-nvswitch-and-all-that", "title": "NVLink, NVSwitch, and all that", "summary": "NVIDIA's NVLink and NVSwitch technologies form a scale-up fabric that connects GPUs tightly enough to behave as a single machine, contrasting with scale-out fabrics like InfiniBand and RoCE that link separate machines. The post explains the physics of high-speed signaling, including differential pairs, SerDes, clock forwarding versus clock and data recovery, and scrambling to maintain signal integrity over short distances.", "body_md": "# NVLink, NVSwitch, and all that\n\nI wrote [before](https://fergusfinn.com/blog/infiniband-roce-rdma/) about\n‘scale-out’ fabrics — [InfiniBand](https://en.wikipedia.org/wiki/InfiniBand),\n[RoCE](https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet), and more\ngenerally\n[RDMA](https://en.wikipedia.org/wiki/Remote_direct_memory_access) — the networks and paradigms that stitch thousands of machines into a datacenter\nisland.\n\nThis post is about the other kind. The names come from an old argument about\nservers: when you need more computer, you can either buy more machines and\n*scale out*, or buy a bigger machine and *scale up*. A scale-up fabric is an\nattempt to connect processors so tightly that they can behave as one. Inside a\nmodern (NVIDIA) GPU machine the fabric is\n[ NVLink](https://en.wikipedia.org/wiki/NVLink)The terms are a bit frustrating really. Because you can scale up your LLM\ninference domain over your scale-out network. And you can certainly scale out inside a\nscale-up domain..\n\n## Fast, or far\n\nLet’s follow the physics of sending bytes over a link. At the lowest level: an\nNVLink is a bundle of wires, set up as [differential\npairs](https://en.wikipedia.org/wiki/Differential_signalling). Each pair is\ndriven by a [SerDes](https://en.wikipedia.org/wiki/SerDes): a circuit that\nturns wide, slow on-chip data into one very fast signal on the wire.\n\nThe signal on the wire is just a voltage changing over time. To turn it back into zeros and ones the receiver has to sample it at the right instants. Before deciding what to put on the wire, we have to decide where the receiver’s sense of time is going to come from.\n\nLength-matching meanders next to a BGA package\n([source](https://i.redd.it/ys3gywkwkr7g1.jpeg)).\n\nOne answer is to send a clock pulse, separate to the data, on its own wire, and\nsample as the clock ticks (a *forwarded clock*). This is how [DDR\nmemory](https://en.wikipedia.org/wiki/DDR_SDRAM) and\n[HBM](https://en.wikipedia.org/wiki/High_Bandwidth_Memory) work. But it only\nworks if the clock’s ticks arrive close enough to all the signal bits for the\nreceiver to sample them cleanly. The receiver can deskew small differences, but\nevery data wire still needs about the same delay and, in practice, about the\nsame length. This is why PCBs are covered in\nsquiggles. At a hundred gigabits per second, one bit occupies about ten\npicoseconds: a millimetre or two of trace. That kind of matching is possible\nacross centimetres, and gets much harder across metres.\n\nWe can instead synchronize each wire individually, sending its clock over each\nsignal wire itself. We know that every transition in the signal (each 0-to-1 or\n1-to-0) marks a bit boundary. So we can use a circuit at the receiver ([clock\nand data recovery](https://en.wikipedia.org/wiki/Clock_recovery)) that keeps a\nlocal oscillator locked to those transitions. The problem is that it works only\nif the transitions keep coming. If you send a million zeros in a row, the\noscillator can start to drift. To keep the data transitioningStatistically, not provably. In principle you can craft data that XORs to\na long run of zeros (“killer packets”, seen in\n[SONET](https://en.wikipedia.org/wiki/Synchronous_Optical_Networking)). Modern\nlinks use long scrambler polynomials to make this impractical., we XOR it\nwith a pseudo-random sequence that both ends can generate\n([scrambling](https://en.wikipedia.org/wiki/Scrambler)).\n\nTop: each edge in the data re-centres the receiver’s clock, but across the run of zeros it free-runs (drawn a few percent fast) and the samples slide off-centre until one lands in the wrong bit. Bottom: the same payload scrambled, the edges never stop.\n\nOnce we can carry a timing signal, we need to put our 1s and 0s on the wire.\nOne of the simplest ways is: high\nvoltage for a 1, low for a 0. That scheme is called\n[NRZ](https://en.wikipedia.org/wiki/Non-return-to-zero)Non-return-to-zero: return-to-zero being a version where you return to\nzero between every signalled bit.\n\nTo signal faster on a single wire under NRZ you have to tick faster. A trace on\na board works like a [low-pass\nfilter](https://en.wikipedia.org/wiki/Low-pass_filter): the copper, and the\ninsulation around it, both absorb more at higher frequenciesThe copper loses because high-frequency current crowds into a thin shell\nat the conductor’s surface (the [skin\neffect](https://en.wikipedia.org/wiki/Skin_effect)). The insulation loses\nbecause the signal’s energy travels through it via the electromagnetic field,\nand the material’s molecules absorb some of it as the field alternates\n([dielectric\nloss](https://en.wikipedia.org/wiki/Dielectric_loss))., adding\nattenuation that the receiver has to make back. It does this with filters that\nre-boost the high\nend, but this boosting amplifies the high-frequency noise along with the\nsignal.\n\nRather than keep paying that rising attenuation, at some point it’s better to\ntry to send more bits per tick.\n[PAM4](https://en.wikipedia.org/wiki/Pulse-amplitude_modulation) signals four\nvoltage levels instead of two, so each tick carries 2 bits of information. The\ncost is that the levels sit a third as far apart. As the level spacing gets\ncloser, errors in decoding start to become more likely.\n\nPAM4 links aim for roughly one bit in ten thousand wrong before correction,\nwhere NRZ links without correction can aim for better than one in a trillion.\nAt these speeds an error rate of one in\na trillion means an error every ten seconds or so, which is rare enough you can\nmanage it at the link layer with checksums and resends. One in ten thousand\nmeans millions of errors a second, which is far too many to handle by resending. So\nPAM4 is usually paired with [forward error\ncorrection](https://en.wikipedia.org/wiki/Error_correction_code). This costs\nlatency, since the decoder has to collect a whole block before it can correct\nanything.\n\nInstead of signalling furiously on a single wire, can’t you just add more\nwires? Yes, up to a point. Fast signals have to leave from the die’s edge: a\nSerDes buried mid-die would have to thread its traces out past all the stuff\naround it. Off-chip bandwidth therefore scales with the die’s perimeter (its\n**shoreline**) while compute scales with its area. The [reticle\nlimit](https://en.wikipedia.org/wiki/Photolithography) caps the die at\nroughly 26 by 33 mm: about 120 mm of shoreline. The [HBM](https://en.wikipedia.org/wiki/High_Bandwidth_Memory)\ninterfaces, NVLink, PCIe, and C2C all compete for that finite edge.\n\nThe other variable is distance. Higher frequency signals travel less far:\npassive copper carries 100 Gb/s per pair about two metres, and 200 Gb/s about\noneThese are IEEE spec objectives: IEEE 802.3ck specifies\n100G-per-lane passive copper to at least 2 m; the follow-on\n[P802.3dj](https://www.ieee802.org/3/dj/index.html) carries a 1 m objective at\n200G per lane, and TE and Intel demoed 224G PAM4 over a 1 m passive cable at\n[OFC 2023](https://opticalconnectionsnews.com/2023/03/ofc-2023-te-intel-demo-224g-pam4-over-passive-copper/).\nLonger is possible with effort: [Marvell](https://www.marvell.com/blogs/224g-long-range-serdes-scale-up-scale-inside.html)\nhas driven 2.5 m of cumulative copper with a long-reach 224G SerDes.. To go further requires extra hardware steps. A *redriver* is an analog\namplifier, and a *retimer* a kind of digital one: a full receiver and\ntransmitter back-to-back decodes the bits and relaunches the signal anew.\n\nThe ledger for all this stuff is energy per bit transferred. Between\nchips on the same board, the channel is clean enough for many slow, simple\nwires: NVLink-C2C, the link between a Grace CPU and its Hopper GPU, has a\nforwarded clock, and moves 900 GB/s for 1.3 pJ/bit. NVLink between GPUs\nuses the SerDes treatment described above, at around 4 pJ/bit: at Blackwell’s\n1.8 TB/s of NVLink bandwidth, roughly 58 W per GPU spent moving bits off the\npackage. Past passive copper, a retimer at each cable end takes it to around\nten, and optics to nearer twenty (the price of optics is the conversion from\nelectrons to light, and is mostly indifferent to the distance\ntravelled) 1.\n\nEnergy per bit against reach, one bar per link technology. These are quite fuzzy, but the direction is correct.\n\nThe binding constraint is where that energy cost is paid. Every joule the links expend is a joule inside the rack’s power and cooling envelope, which competes with compute. For now that means passive copper, and therefore scale-up stays within a rack.\n\n## How we got here & where we’re going\n\nReading another chip’s memory with an ordinary load is an old idea for CPUs.\nSGI’s [NUMAlink](https://en.wikipedia.org/wiki/NUMAlink) stitched as many as\n512 processors into a single memory space by 2001, and coherent\n[HyperTransport](https://en.wikipedia.org/wiki/HyperTransport) did the same for\nthe handful of sockets on an Opteron board. Commodity clusters killed the big\nmachines, but the board-scale version is everywhere: every multi-socket\nserver is still a small\n[NUMA](https://en.wikipedia.org/wiki/Non-uniform_memory_access) machine.\n\nOn the GPU side, NVIDIA added a single virtual address space across GPUs, with\npeer-to-peer access over PCIe, in CUDA 4.0 (2011)[CUDA 4.0](https://developer.nvidia.com/blog/unified-memory-in-cuda-6/)\nintroduced unified virtual addressing and GPUDirect peer-to-peer on Fermi.\nWhether two GPUs can map each other’s memory depends on the bus between them;\nover PCIe it also requires both devices to sit under the same root complex.. By 2016 a gen3 x16 slot\nmoved 16 GB/s per direction against 720 GB/s of local HBM2: one forty-fifth the\nbandwidth. NVLink was\nintroduced with the P100 that year to try to widen the path: four links per\nGPU, with 160 GB/s of aggregate bidirectional bandwidth. The first generation\nalso joined CPUs to GPUs: IBM sold POWER8 servers with [the CPUs wired to P100s\nover NVLink](https://developer.nvidia.com/blog/inside-pascal/).\n\nFour links can’t fully connect eight GPUs, so the DGX-1 wired its P100s into a\n[hybrid cube mesh](https://developer.nvidia.com/blog/inside-pascal/), each GPU\ndirectly linked to four of the others. Transfers took one hop or two depending\non the pair, and NCCL threaded its rings around the physical wiring. In 2018\nNVIDIA put a switch between the GPUs instead:\n[NVSwitch](https://developer.nvidia.com/blog/nvswitch-accelerates-nvidia-dgx2/),\nan 18-port chip that carries traffic between all of its ports at the full\nlink rate simultaneously. The DGX-2 holds twelve of them, and any pair of its\nsixteen V100s can transfer at the full 300 GB/s, with all sixteen GPUs\ncommunicating at once.\n\nThe third-generation switch (with Hopper, in 2022) carried ALUs for in-network\nreductionNVIDIA’s name for this is\n[SHARP](https://docs.nvidia.com/networking/display/sharpv300), the Scalable\nHierarchical Aggregation and Reduction Protocol. It came over with the 2020\nMellanox acquisition: Mellanox’s InfiniBand switches had been doing in-network\nreductions for MPI collectives since 2016.:\nin an allreduce, every GPU sends its data into the switch once and receives\nthe sum once, instead of shuttling partial results around a ring. That halves\nthe bytes each GPU moves, and roughly doubles the effective bandwidth of the\ncollective.\n\nThe\n[GH200](https://nvidianews.nvidia.com/news/nvidia-grace-hopper-superchips-designed-for-accelerated-generative-ai-enter-full-production),\nin production since mid-2023, brings the CPU into the domain. It’s a Grace CPU paired with\na Hopper GPU over NVLink-C2C, with the CPU’s 120 GB of LPDDR5X and the GPU’s 96 GB of\nHBM in one address space.\n\nThe GB200 followed in 2024, with one Grace for every two Blackwell GPUs. The\n[NVL72](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) is 36 of them\nwired into a rack, the 72 GPUs and 36 CPUs joined through eighteen switch\nchips and a copper spine, 130 TB/s of any-to-any bandwidth between them.\nNVIDIA calls the rack “a single, massive GPU”. A thread on any of the 72 can\nissue a load against any byte of its 13.4 TB of HBM.\n\nThe link generations so farPair counts are per direction; per-GPU figures are NVIDIA’s bidirectional aggregates.:\n\n| gen | GPU | year | per pair | pairs per link | links | per GPU |\n|---|---|---|---|---|---|---|\n| 1 | P100 | 2016 | 20 Gb/s NRZ | 8 | 4 | 160 GB/s |\n| 2 | V100 | 2017 | 25 Gb/s NRZ | 8 | 6 | 300 GB/s |\n| 3 | A100 | 2020 | 50 Gb/s NRZ | 4 | 12 | 600 GB/s |\n| 4 | H100 | 2022 | 100 Gb/s PAM4 | 2 | 18 | 900 GB/s |\n| 5 | B200 | 2024 | 200 Gb/s PAM4 | 2 | 18 | 1,800 GB/s |\n\nNVIDIA has also named the next three stepsFrom [NVIDIA’s Vera Rubin\nmaterial](https://developer.nvidia.com/blog/nvidia-vera-rubin-pod-seven-chips-five-rack-scale-systems-one-ai-supercomputer/)\nand its GTC 2026 roadmap. Blank cells are numbers NVIDIA hasn’t yet put\nagainst the names.:\n\n| NVLink | system | promised | per pair | per GPU | domain |\n|---|---|---|---|---|---|\n| 6 | Vera Rubin NVL72 | 2026 | 200 Gb/s PAM4, both directions | 3.6 TB/s | 72 GPUs, one rack |\n| 7 | Rubin Ultra NVL576 | 2027 | — | — | 576 GPUs, eight racks |\n| 8 | Feynman | 2028 | — | — | 1,152 GPUs, eight Kyber racks, optical between racks |\n\nVera Rubin keeps the rack as it is — 72 GPUs, a spine of five thousand copper\ncables — and doubles the bandwidth through it, to 260 TB/s in aggregate. The\ntick rate and the modulation are unchanged from NVLink 5, and the spine has no\nroom for more cables. The doubling comes from each pair carrying both\ndirections at once.\nBoth ends transmit onto the same two wires at the same time, so the voltage at\neach receiver is the sum of two signals, one of which, its own, the receiver\nalready knows. It synthesizes a copy of what its transmitter is sending and\nsubtracts it; what remains is the far end’s signal. The subtraction is a timing problem: a tick\nlasts under ten picoseconds, and the copy has to line up with the actual echo\nto within a fraction of that, or the residue swamps the far end’s\nsignalFrom [SemiAnalysis’s Vera Rubin\nanalysis](https://newsletter.semianalysis.com/p/vera-rubin-extreme-co-design-an-evolution);\nNVIDIA’s own material stops at the aggregate numbers. Gigabit Ethernet has\ndriven its four twisted pairs this way since 1999, at a tick rate nearly a\nthousand times slower..\n\nThe NVL576 is the first NVLink domain bigger than a single rack: eight of\nthem, with copper and optical links between the racksNVIDIA’s words: eight MGX NVL racks combined “with copper and direct\noptical connections”.. Kyber is the rack\ndesign that succeeds the 72-GPU one, the one the Feynman-era NVL1152 is built\nfrom. It fits 144 GPUs in one cabinet, and\nswaps the cable spine for a midplane, one large circuit board that the GPU\ntrays plug into from both sides, the links running as traces inside it. In\nJuly 2026 SemiAnalysis reported that the board, at 78 layers, was proving\nhard to manufacture, and that the rack had slipped to 2028Reported via [Tom’s\nHardware](https://www.tomshardware.com/pc-components/gpus/nvidias-kyber-rack-for-rubin-ultra-slips-to-2028),\nwhich also reports the NVL72x2 stopgap rack cancelled and NVL576 volumes in\ndoubt.. NVIDIA’s\nresponse was that the roadmap is intact.\n\n## Other sides of the scale-up story\n\nMemory semantics are wired into NVIDIA’s answer to scale-up. But they’re not everyone’s. Google demonstrates that compiler-scheduled communication can create a tightly coupled scale-up system by another route.\n\nGoogle connects its TPUs directly to one another, with no switch between them.\nIts network is a torus (most recently, a 64-chip rack arranged as a 4×4×4 cube),\nwith 1.2 TB/s of ICI (inter-chip interconnect) per chipThe figures are\n[Ironwood](https://docs.cloud.google.com/tpu/docs/tpu7x)’s, the seventh\ngeneration, which scales to 9,216 chips. The torus dates to TPU v2’s 16×16\ngrid in 2017; the third dimension and the optical switching arrived with [TPU\nv4](https://arxiv.org/abs/2304.01433).. Unlike NVIDIA’s\nmodel, a TPU doesn’t have an instruction to load from another chip’s memory.\nCommunication is written explicitly into the program, and\n[XLA](https://openxla.org/xla) schedules the collective operations alongside\nthe computation at compile time. The participants and tensor shapes of a\ntraining step’s collectives are known before the program runs, and XLA uses\nthat foreknowledge to overlap communication with compute and keep the direct\nlinks busy.\n\nBetween racks, the cubes are joined by optical circuit switches: mirrors that\nhold a light path open from one cube to another. Google repositions the mirrors\nbetween jobs, giving each job a torus of its own size and mapping out failed\nracksThe [TPU v4 paper](https://arxiv.org/abs/2304.01433) gives the\nreasons: cubes enter service before the whole pod exists, and a job can\nrequest variant topologies, including twisted tori for all-to-all traffic..\n\nIn an MoE model at inference time, the expert layers send each token to\ndifferent chips chosen at runtime. On a torus and some reasonable MoE domain it\ncan end up crossing some sixteen hops before finding its expert. The eighth TPU\ngeneration, announced in April 2026, forked the topology choices. TPU 8t, for\npre-training, keeps the torus and grows it to 9,600 chips. TPU 8i, for\ninference, replaces the torus with “Boardfly”, 32-chip groups fully connected\nby copper, up to 36 groups joined through the same optical circuit switches,\nand any pair of the pod’s 1,024 active chips within seven hops[Google’s technical deep\ndive](https://cloud.google.com/blog/products/compute/tpu-8t-and-tpu-8i-technical-deep-dive).\nThe name is after\n[Dragonfly](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/34926.pdf),\nthe 2008 high-radix topology it adapts..\n\nOn the open side of runtime dynamism: AMD, Broadcom, Intel, Google, Meta,\nMicrosoft, and others formed the [UALink consortium](https://ualinkconsortium.org/) in\n2024 to specify an open equivalent of NVLink. The [1.0\nspecification](https://ualinkconsortium.org/blog/ualink-200g-1-0-specification-overview-802/),\npublished in April 2025, defines 200 Gb/s lanes, domains of up to 1,024\naccelerators, and reads, writes, and atomics against remote memory in\ntransactions of 64 to 256 bytes. But no UALink switch has shipped as of\nmid-2026. The first parts are due in labs late this year, and in products in\n2027. In April 2026 the consortium published a 2.0 specification[The Register, April\n2026](https://www.theregister.com/2026/04/07/ualink_2_specs/); the first\nswitch vendors, [Upscale AI and Astera\nLabs](https://www.hpcwire.com/2025/12/02/upscale-ai-eyes-late-2026-for-scale-up-ualink-switch/),\nboth target late 2026.. AMD’s\nHelios racks (72 MI400-series GPUs, due in the second half of 2026) will\ncarry UALink tunnelled through Broadcom Tomahawk 6 Ethernet switches in\nthe meantimeReported by [SemiAnalysis](https://newsletter.semianalysis.com/p/amd-advancing-ai-mi350x-and-mi400-ualoe72-mi500-ual256) and [The Next Platform](https://www.nextplatform.com/compute/2026/02/23/amd-says-helios-racks-and-mi400-series-gpus-on-track-for-2h-2026/4092199).\nAMD quotes 260 TB/s of scale-up bandwidth for the rack, the same figure\nNVIDIA quotes for Vera Rubin..\n\nThere’s also an effort that parallels the rise of RoCE, an attempt to use\nEthernet to dethrone a proprietary fabric. In\nJuly 2025 Broadcom shipped [Tomahawk\nUltra](https://investors.broadcom.com/news-releases/news-release-details/broadcom-ships-tomahawk-ultra-reimagining-ethernet-switch-hpc),\na 51.2 Tb/s Ethernet switch chip aimed at NVSwitch. It is pin-compatible with\nthe Tomahawk 5 that runs ordinary datacenter networks, with the Ethernet protocol\nmodified for scale-up: cut-down frame headers, link-layer retries, lossless\ncredit-based flow control (à la InfiniBand), and collectives executed inside the switch[ServeTheHome’s coverage](https://www.servethehome.com/broadcom-tomahawk-ultra-launch-for-scale-up-ethernet/)\nhas the details. The in-switch collectives include AllReduce and\nBroadcast: the SHARP feature set, in Ethernet silicon..\nPort to port, it takes 250 nanoseconds. Broadcom had been in the group that\nannounced UALink in May 2024, and did not take a seat when the consortium’s\nboard formed that OctoberThe [May 2024 group](https://www.businesswire.com/news/home/20240530653602/en/AMD-Broadcom-Cisco-Google-Hewlett-Packard-Enterprise-Intel-Meta-and-Microsoft-Form-Ultra-Accelerator-Link-UALink-Promoter-Group-to-Drive-Data-Center-AI-Connectivity)\nwas AMD, Broadcom, Cisco, Google, HPE, Intel, Meta, and Microsoft. The\n[October 2024 incorporation](https://www.businesswire.com/news/home/20241029998800/en/Ultra-Accelerator-Link-Consortium-Incorporates-Announces-Membership-Opportunity)\nlists a board of AMD, Astera Labs, AWS, Cisco, Google, HPE, Intel, Meta, and\nMicrosoft. Broadcom remains an ordinary member; its stated position, [per The\nRegister](https://www.theregister.com/2025/07/15/broadcom_ethernet_scale_up/),\nis that Ethernet can do the job now rather than waiting for the UALink\nprotocol.. In October 2025 the Open Compute Project opened\na workstream,\n[ESUN](https://www.opencompute.org/blog/introducing-esun-advancing-ethernet-for-scale-up-ai-infrastructure-at-ocp),\nto standardize Ethernet framing and switching for scale-up, seeded with\nBroadcom’s [Scale-Up\nEthernet](https://www.broadcom.com/blog/tomahawk-ultra-ethernet-to-the-rescue-for-hpc-and-ai-scale-up)\nspecification. Its twelve founding members include both Broadcom and\nNVIDIAThe full list: AMD, Arista, Arm, Broadcom, Cisco, HPE, Marvell, Meta,\nMicrosoft, NVIDIA, OpenAI, and Oracle..\n\nWith [NVLink\nFusion](https://nvidianews.nvidia.com/news/nvidia-nvlink-fusion-semi-custom-ai-infrastructure-partner-ecosystem),\nannounced in May 2025, NVIDIA is trying to license access to the NVLink fabric.\nCustom CPUs attach to NVIDIA GPUs over NVLink-C2C; or custom accelerators\nintegrate an NVLink chiplet and join the switched domain. Fujitsu, Qualcomm, and Intel\nhave agreed to attach CPUs; AWS’s Trainium4 will take the accelerator path,\njoining the NVLink 6 domain through the same switch trays as Vera Rubin[The Fusion\nannouncement](https://www.tomshardware.com/pc-components/cpus/nvidia-announces-nvlink-fusion-to-allow-custom-cpus-and-ai-accelerators-to-work-with-its-products),\n[the Intel\nagreement](https://nvidianews.nvidia.com/news/nvidia-and-intel-to-develop-ai-infrastructure-and-personal-computing-products),\nand [the Trainium4\nintegration](https://developer.nvidia.com/blog/aws-integrates-ai-infrastructure-with-nvidia-nvlink-fusion-for-trainium4-deployment/)..\n\n## Who wins?\n\nThe previous post was about the scale-out version of this contest:\n[InfiniBand](https://fergusfinn.com/blog/infiniband-roce-rdma/) against RoCE,\na proprietary integrated fabric against commodity Ethernet adapted to the\njob, and the commodity side continues to win ground on cost and ubiquity. The\nscale-up battles have now started.\n\nUnlike scale-out, there’s no clear commodity candidate to sweep the hyperscaler market and dethrone NVIDIA. NVLink is one of NVIDIA’s strongest assets: it’s taken them to rack-scale in a tightly integrated, all-to-all, ultra fast fabric years before anyone else. They got there by betting early and betting hard on integration. This makes it much more difficult for anyone else to push in and rebuild scale-up from the ground up & sell it to customers. Over the next few years, we’ll start to find out if scale-up is really so different.\n\n## Footnotes\n\n-\nThe only first-party number is C2C’s: NVIDIA quotes 1.3 pJ/bit in its\n\n[Grace Hopper deep dive](https://developer.nvidia.com/blog/nvidia-grace-hopper-superchip-architecture-in-depth/). The single-ended, forwarded-clock design is described in NVIDIA’s[ISSCC 2023 paper](https://ieeexplore.ieee.org/document/10067395/). Nobody publishes one for NVLink itself; the ~4 pJ/bit is[Marvell’s figure](https://www.marvell.com/blogs/224g-long-range-serdes-scale-up-scale-inside.html)for their long-reach 224G SerDes, the class of part these links are built from. The last two rungs are worked back from the power draw of shipping 800G parts: Credo’s[product brief](https://credosemi.com/wp-content/uploads/Credo_AEC_800G_SPAN_O_O_RHS_Product_Brief_2025Sept26.pdf)for its 800G AEC specifies a typical 10 W dissipation, and lengths up to 7 m — over 800 Gb/s, about 12 pJ/bit; a pluggable optical module burns 13—18 W, 16—22 pJ/bit.[↩](#user-content-fnref-1)\n\n[Suggest an edit](https://github.com/fergusfinn/blog/edit/main/src/content/blog/nvlink-scale-up.mdx)\n\nLast modified: 21 Jul 2026", "url": "https://wpnews.pro/news/nvlink-nvswitch-and-all-that", "canonical_source": "https://fergusfinn.com/blog/nvlink-scale-up/", "published_at": "2026-07-21 00:00:00+00:00", "updated_at": "2026-07-21 10:56:23.077632+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-chips"], "entities": ["NVIDIA", "NVLink", "NVSwitch", "InfiniBand", "RoCE", "RDMA", "HBM", "DDR SDRAM"], "alternates": {"html": "https://wpnews.pro/news/nvlink-nvswitch-and-all-that", "markdown": "https://wpnews.pro/news/nvlink-nvswitch-and-all-that.md", "text": "https://wpnews.pro/news/nvlink-nvswitch-and-all-that.txt", "jsonld": "https://wpnews.pro/news/nvlink-nvswitch-and-all-that.jsonld"}}