Hello again, and welcome back to Fast Company’s Plugged In.
Last week, I attended an Nvidia media event that included a field trip to an undisclosed location in Silicon Valley—a building without any identifying signage. Though hundreds of racked GPUs were chugging away at real AI tasks inside, Nvidia calls the facility an “engineering superlab” rather than a data center. Among the AI being crunched during our visit: a test version of OpenAI’s GPT model optimized for Nvidia’s new Vera Rubin platform, named after the legendary astronomer.
For show-and-tell purposes, several powerful computers were on display in the building’s lobby. Containing GPUs, CPUs, and supporting components and known as trays, they slide into racks like the ones in the superlab. Data centers equipped with vastly larger quantities of these racks power a huge percentage of the AI in our lives. Basically, we all use these Nvidia tray computers all day long. It’s just that under normal circumstances, we never see them or even have much reason to give them any thought. Getting a close-up peek at the ones at Nvidia’s unmarked superlab was a rare opportunity to contemplate how they do what they do.
For the sake of comparison, the ones in the lobby included a Vera Rubin tray sittting next to one based on Nvidia’s previous-generation platform for its GB200 and GB300 chips. At a glance, it was obvious that the redesign went far beyond the new tray’s more powerful chips. The older tray looked like a rat’s nest, with an array of cables snaking every which way to connect components. The Vera Rubin tray was radically more streamlined. It was cable-free. Well, not literally cable-free. “We call it cableless—it actually has two cables,” explained Senior VP of Hardware Engineering Andrew Bell, a 25-year Nvidia veteran. One cable remains to hook up the tray to power, another to connect it to networking resources. But they were unobtrusive enough that I didn’t notice them until he pointed them out.
It also might be unfair to call the earlier design a rat’s nest. it was way more fastidiously organized than the bedside tangle of USB cables I use to charge my phone, laptop, and iPad. But when a computer has 50 or 60 cables, as this one did, it gives off an overwhelming impression of steampunk-y chaos. Especially when it’s sitting next to one that’s so close to being cable-free.
Two overarching questions are relevant to this redesign: Why did Nvidia decide to go (almost) cableless, and how did the company pull it off? The first answer might be pretty obvious if you’ve ever built a desktop PC, or at least opened one up.
Inside a computer, cables just complicate things. Some 50 or 60 cables add up to 100 to 120 additional steps during assembly. Each connection is a place something could go wrong during the building process and, after that, a potential point of failure. Cables create thickets that get in the way of other components, both during manufacturing and later servicing.
“It takes two to three hours to assemble [the GB200/GB300 tray],” Bell said. “And when we first started building it, you had about a 20% chance of getting it right the first time.”
Ian Buck, VP and GM of hyperscale and high-performance computing, added, “Just imagine yourself plugging in every one of those cables and trying to do it at a scale of Nvidia.”
The success rate did gradually increase, but it wasn’t just Nvidia’s own scale that was an issue. Rather than mass-producing trays itself, the company shipped parts to contractors and manufacturing partners, such as Dell. Each time production expanded, it set off a Groundhog Day-style cycle as new teams learned how to assemble the cable-heavy trays.
If cables’ downsides are so many and varied, why have they been so fundamental to computer designs for so long? One way or another, components do need to talk to each other. For a long time, it wasn’t obvious how to achieve that *sans* cabling.
For Nvidia, the breakthrough cascaded out of another major design achievement reflected in the Vera Rubin platform: It’s fanless. Instead of using air cooling to prevent overheating, the tray is 100% liquid cooled, using a new approach that works with water at up to 113 degrees Fahrenheit, dramatically reducing energy requirements compared to earlier liquid-cooling techniques.
As a bonus, removing fans also promises to dramatically reduce the noise pollution inside data centers. (At the superlab, which is full of fan-cooled racks, we wore earplugs, and Bell spoke to us with a microphone despite being only a few feet away.)
When Nvidia figured out how to ditch the fans, it opened up room inside the Vera Rubin trays. More room allowed the company to create a midplane connector, based on its NVLink technology, that established cable-free communications between components. Even if Nvidia had somehow found room for the midplane connector in an air-cooled tray, it wouldn’t have worked: It would have obstructed airflow.
In short, one elegant technical breakthrough enabled another. You don’t need to get that backstory from Nvidia executives to see the elegance in the resulting Vera Rubin tray. It’s sleek, black, and bordering on brutalism in its lack of ornamentation. Compared to it, its predecessors look like highly compressed versions of 1951’s Univac.
All of this is ultimately in service of a faster, more reliable ramp-up of the Vera Rubin platform, in factories that can now be more fully automated. “This [tray] can be assembled in five minutes instead of two to three hours,” Bell said. “The first time you put it together, you have a 95% chance of getting it right. You can take it to a new factory. The first time they put it together, it’s 95%. We expect after a month, they’ll be at 98% or 99%.”
With other AI chip makers ever more intent on stealing market share from Nvidia—hello, AMD!—those efficiencies matter. But even if Nvidia wasn’t after elegance for elegance’s sake, the company has achieved it—and I, for one, will never assume that cables are one of computing’s eternal necessary evils again.
You’ve been reading Plugged In*, Fast Company’s weekly tech newsletter from me, global technology editor Harry McCracken. If a friend or colleague forwarded this edition to you—or if you’re reading it on fastcompany.com—you can check out previous issues and sign up to get it yourself every Friday morning. I love hearing from you: Ping me at hmccracken@fastcompany.com with your feedback and ideas for future newsletters. I’m also on Bluesky, Mastodon, and Threads, and you can *
Trump Media’s $100k-a-month Truth Social scheme is the president’s most desperate grift yet Truth API is what happens when a historically unpopular lame-duck president needs to lock in a revenue stream before he leaves office. Read More →
Stripe is the most underrated—and influential—$159 billion company in the world 20 Stripe employees turned founders reveal the secrets of its power. Plus: The largest public dataset of Stripe alums who’ve started companies. →The amazing thing about Samsung’s new folding phones is their normalcy
After seven years of iteration, a Galaxy Z Fold is just a flagship phone that unfolds—not a weird category unto itself. Read More →
[An OpenAI model went rogue on the internet and stole test answers](https://www.fastcompany.com/91578008/an-openai-model-went-rogue-on-the-internet-and-stole-test-answers)
The unprecedented incident took place during a safety test at OpenAI. [Read More →](https://www.fastcompany.com/91578008/an-openai-model-went-rogue-on-the-internet-and-stole-test-answers)
[1,273 designers tell all: Their rates, real pay, and how they really feel about AI](https://www.fastcompany.com/91571502/freelance-design-pricing-transparency-project-report)
In an exclusive new report from Fast Company and AIGA, freelance designers share details about their work lives. →
The most useful ways to connect your apps to ChatGPT and Claude AI assistants can now search your notes, organize your inbox, update your calendar, edit images, and work across the services you use every day. Read More →