Not So Fast On That Charge For 800 Volt Datacenter Power

Schneider Electric executives predict no more than 10 percent of new AI nodes will adopt 800-volt power by 2030, despite Nvidia pushing the high-voltage architecture for its Kyber rack design targeting 1-megawatt racks. The company's technical director Rob Bunger said at a datacenter event that initial sidecar solutions will deploy by late 2024, with centralized power distribution expected by 2029, as conventional 48-volt in-rack power becomes impractical beyond 400 kilowatts due to cable congestion and space constraints.

Not So Fast On That Charge For 800 Volt Datacenter Power Nvidia might be pushing its customers towards adopting 800 volt power in their datacenters, but Schneider Electric executives don’t expect more than 10 percent of new AI nodes to move to this high voltage anytime soon. Companies are already packing in racks at 140 kilowatts and increasingly at 200 kilowatts. But Nvidia’s “Kyber” rack architecture, which envisages the fabled 1 MW rack, is predicated on 800 volts. At a datacenter event in Buffalo, New York, Schneider Electric’s cloud and service provider technical and solutions director Rob Bunger explained that conventional in-rack power is at 48 volts, which is “touch safe.” But mechanical/electrical issues start kicking in as racks head to 400 kilowatts, Bunger continued. This includes congestion in the rack from feeds and converters and the power capability of the rack of the bus bar at low voltage. As Bunger explained, a 150 kilowatt rack running 72 GPUs would need eight “whips” or power cables. A 1,000 kilowatts rack running 144 GPUs would need 32 whips – and they would be much larger. “Which is impractical. It doesn’t make any sense at all,” says Bunger. So, Bunger continued, “The two things you want to do is one, move those converters –rectifiers turning AC into DC or stepping DC voltages – out of the rack. You consolidate them and you up the voltage to get more power per wire.” Getting to 800 volts will initially be through sidecars in the rack itself, he said, but that will itself use up rack space. “It starts to become a little bit impractical to do a ton of side cars as the datacenter gets larger and larger, so there will be centralized solutions that allow you to do larger chunks of power conversion to do multiple routes, call it centralized DC distribution.” But he said, “Now you have to think about, ‘Okay, I have potentially the conversion happening outside of the white space, and I have to get that 800 volts to the IT rack,’ so that you have the power distribution and protection and the sensitivity, and all this stuff.” He predicted small scale deployments by the end of this year, with “more centralized” power distribution solutions by 2029. Initial sidecar solutions would “be able to provide 600 kilowatts to 1 megawatt worth of power, so you can imagine that's going to be being able to power one, two, or three of these very high-end AI racks.” For the centralized solutions, “We'll have converters that'll be able to take AC to 800 volts, and these will come in chunk sizes of about two to five megawatts, roughly, and that's what we see in the middle.” “You see the term DC UPS,” he added, “So there would be energy storage batteries coupled with these. So from a perspective of uninterrupted power of DC output, it would be considered the DC UPS. The sidecar again can be configured with energy storage.” But while Nvidia is ploughing ahead with 800 volts, Bunger said this doesn’t mean large scale retrofitting of existing sites. “We think by 2030 maybe 10 percent of the new AI nodes coming out will be required to have 800 volt DC.” And even brand new datacenters will not be entirely based on 800 volts. “We'll likely see pockets of it within datacenters,” Bunger said, together with a mix of medium to low voltage. Nevertheless, he said, this would still have an impact on overall datacenter design as density increases. “The ratio of facility equipment to IT equipment is definitely changing.” Unsurprisingly, Schneider said a holistic approach to reworking power infrastructure made sense – or a single vendor to put it another way. “You really have to think of the holistic solution from the panel boards, the bus waves, the circuit breakers.” “So, the first phase of seeing 800 volts getting deployed will be a very highly engineered full system design to make sure it's very reliable. As the whole market matures, both from a standards perspective, and more options, we will start to see more interoperability between products.” Things are further complicated by the nature of AI workloads compared to traditional cloud workloads, Bunger added. Traditionally, “The whole purpose of the facility was to protect the IT load, you know, high nine, high safety, and high nines of availability” AI workloads, by comparison, are not static. “Especially with AI training. They pulse, and so you have to think about what that effect is on the grid, pulsing load, and then very, very large loads.” Fault ride through becomes an issue at this point. “So not only does the datacenter facility have to protect the IT load historically, it now also have to protect the grid at the same time.” Schneider chief marketing officer Kevin Brown added that there was nothing particularly magical about 800 volts, “The reason the industry is going to 800 volt DC is because of EVs. Because there's a supply chain available to deliver connectors, and there's an understanding of how it works, and so forth.” If changing power architecture wasn’t enough to keep AI and HPC engineers busy, Schneider Electric also argued that the shift to liquid cooling as these power hungry racks proliferate will help AI datacenters flush out more tokens. And the company pushed back on claims that AI datacenters are causing a water crisis. Tuan Hoang, Schneider’s head of cooling technology and product development, outlined a case study of water consumption by putative AI datacenters in Texas and Paris based on the company’s reference designs, comparing the impact of evaporative cooling versus liquid cooled systems. The study compared the impact of running a 100 megawatt datacenter in Texas and Paris. Based on these, he said, a 100 megawatt air-cooled/evaporative tower site in Dallas would deliver a 1.148 PUE and consume 161 Olympic swimming pools of water, while delivering 1.99x108 tokens. Shift that to more temperate Paris, and the PUE comes in at 1.11, with the same water consumption but 2.78x108 tokens delivered. Here is the comparative data for the Dallas scenario: And here it is for the Paris scenario: And finally, here is a summary table comparing the two: A datacenter in Texas, using liquid cooling with Vera-Rubin AI systems from Nvidia, would deliver a 1.04 PUE, consume 79 pools of water, and generate 2.52x1011 tokens. Its Paris-based equivalent would deliver 1.04 PUE, consume 20 pools of water, and generate 2.91x1011 tokens. Just for comparison, an Olympic pool equates to 2,500 m3 of water, while a typical Dallas household consumes 416 m3 of water a year, and a Paris household sloshes through 110 m3 a year. Of course, the real challenge is getting permits to squeeze in an AI scale datacenter in the compact, power constrained City of Lights compared to power and land abundant Texas. But Huang said the exercise was significant give that water consumption is one of the reasons communities are pushing back against datacenters. He said the numbers showed that “water consumption is a choice, not a requirement.” Using water to extract heat more efficiently comes at a cost, he said. “But the location decision also changes the amount of water that's consumed. Imagine extrapolating that this out to the extreme to go to the northern or cooler environment, or you go to the Middle East, or desert environment.” And Schneider CMO Brown added, the industry had to do a better job explaining that liquid cooling “does not mean water use.” Rather, water use is down to the use of evaporative cooling, which was adopted to get better energy efficiency. But AI datacenters are effectively synonymous with liquid cooling. By adopting liquid cooling, Brown said “because the temperatures change, you have unique opportunities with that architectural change, our water use goes down nominally, relatively speaking. The amount of kilowatts you're consuming and tokens you're generating will become much more efficient.” The myth that AI data centers, because they're liquid cooled, are consuming water, is “not true, you can avoid it and still have that efficient data center.” Though local communities are likely to have multiple other objections to having a 200 acre plus AI facility dropped on them.