Neoclouds reshape traditional architectures to meet AI demands
The rise of neoclouds — AI-first cloud providers — is transforming enterprise infrastructure.
The unique requirements of AI models and agents have paved the way for new collaborations between hardware companies, data platforms and neoclouds. As the industry moves from training to inference, these partners are striving to meet the demands for compute power and performance.
“Agentic AI is moving into production and inference is becoming the dominant workload,” said Junxia Zhou (pictured, bottom row, left), senior product manager at Super Micro Computer Inc. “Inference is highly sensitive to latency, so neocloud providers deploy the AI closer to end users. If you are deploying the infrastructure at that scale … you need to use a proven design or infrastructure configuration that can be replicated across multiple locations.”
Zhou, alongside Phil Manez (top row, left), vice president of strategic initiatives at Vast Data Inc.; Anders Graham (top row, right), senior director of SSD marketing and business development at Kioxia Holdings Corp.; and Omar Lari (bottom row, right), senior director of products, infrastructure as a service, at Crusoe Inc., spoke with theCUBE Research’s Rob Strechay for the Supermicro Open Storage Summit interview series, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed their collaboration, the constraints of current enterprise infrastructure and how agentic AI is reshaping the industry. ( Disclosure below.)*
Neoclouds sneak up on hyperscalers
Supermicro works with Crusoe, a neocloud that takes an “energy-first” approach to its services. The company draws on unique sources of energy, including wind and natural gas, for its data centers and stacks AI infrastructure services on top of that — all in the name of efficiency.
“Our main goal is to keep this as simple and straightforward as possible,” Lari said. “We don’t want to have a lot of bloat and have 10 different services that do all of the same things. We want customers to be able to run their training jobs, quickly convert that from research projects into revenue-generating inference projects, and be able to go out into the market and repeat that cycle over and over again.”
Neoclouds have an edge on the hyperscalers because they have an AI-first design with none of the baggage from legacy architectures. Crusoe partners with Vast to leverage their disaggregated shared-everything architecture (DASE) to provide a high-performance, low-latency storage platform with more flexibility than traditional clouds.
“The rise of neoclouds is really strong validation that you can’t run AI on legacy architectures,” Manez said. “The hyperscalers have a lot of technical debt. It’s really hard to flip that around. And the path to success, if you want to move aggressively into AI, is to go to someone who knows how to build this type of infrastructure. It has to be purpose-built for AI or it’s just never going to scale.”
Crusoe also uses Kioxia’s flash memory and solid-state drives to increase power efficiency. The combined demand for flash production and enterprise SSDs exceeds demand from mobile and client applications for the first time, years earlier than expected, Graham noted.
“Neoclouds such as Crusoe, they’re driving such unprecedented demand,” he said. “They’re really building out this infrastructure purpose-built for AI. These large language models, they’re the key demand drivers … and a lot of this is going to the neoclouds as they’ve been more efficient at building out this infrastructure.”
AI infrastructure battles bottlenecks
Two of the biggest constraints on the agentic AI trend are memory and storage. Customers need to be strategic about how they deploy their hardware, according to the panel. High-capacity quad-level cell technology will ease those bottlenecks, according to Graham, while Zhou sees a shift toward liquid cooling in AI data centers.
“Power availability is a very challenging issue for us,” Zhou said. “The power capacity is always the first consideration whenever a customer is building their data infrastructure. As the modern GPU power consumption continues to increase, the air cooling alone is becoming difficult and expensive at scale. Customers are adopting liquid-cooling technology because it will improve the power efficiency and enable higher rack density and allow operators to lower the operating cost.”
Lines have started to blur between compute, storage and infrastructure, according to Lari. Collaborators will have to keep reevaluating which combination of those elements gets customers the most bang for their buck.
“We’re just constantly playing this high stakes game of poker and making sure that we’re putting the bets in the right places,” he said. “Making sure we’re working with our vendors, with Supermicro and Vast and Kioxia, to make sure we’ve got storage when we need it at the perfect amount of time because the other thing we don’t want to do is have idle resources sitting there, not generating revenue.”
Stay tuned for the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Supermicro Open Storage Summit interview series.
( Disclosure: TheCUBE is a paid media partner for the Supermicro Open Storage Summit interview series. Neither Supermicro, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)*
Photo: SiliconANGLE
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more** 11.4k+ theCUBE alumni**— Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.