LAS VEGAS – Oracle Cloud’s Acceleron networking and infrastructure technology platform gained new capabilities targeted at strengthening AI data movement, distributed AI communication, and secure multitenant AI infrastructure.
For AI data movement, Acceleron has a new fabric designed to help users deploy inference, storage, and high-performance compute (HPC) workloads through remote direct memory access (RDMA) connectivity on Oracle Cloud Infrastructure (OCI) front-end compute. There is also an expanded non-volatile memory express (NVMe) block-storage attachment option to help support AI, database, and other data-heavy workloads across Acceleron bare metal and virtual machine (VM) architectures. Distributed AI communication updates include a collective communication library that helps users keep distributed AI workloads running when network paths fail. It does this by separating collective traffic across OCI’s backend multiplanar network and directing traffic toward “healthier” paths should links, ports, or network interface cards (NICs) fail.
There is also a new transparent front-end multiplanar networking feature that improves resiliency and provides congestion control for front-end RDMA traffic.
Oracle unveiled Acceleron at its annual AI World event last year, touting the platform as spanning fabric network architecture, fabric accelerators, and host network accelerators. It leverages dedicated network fabrics and direct paths to minimize latency and variability to unify and streamline complex cloud network functions, storage, and security.
Pradeep Vincent, EVP for compute and AI infrastructure at OCI, explained during an interview with SDxCentral at this week’s Oracle AI World event that customers have already started to see a “night and day difference” in using the platform.
“Particularly for complex, very large cluster training runs, we see a night and day difference between accelerant multiplanar versus the traditional cluster networking,” Vincent said, adding, “it's not even comparable.”
Vincent did note that the latest updates take that further by providing similar RDMA capabilities on the front end, “which brings massive amounts of challenges.”
“Now in the front end it needs to coexist with other non-RDMA traffic,” Vincent continued. “In the past that was very challenging. Now with Acceleron we have the right technology investments to actually get us there.”
Networking resiliency routes #
That investment is also helping boost network resiliency around what Vincent said are increasingly complex network interface architectures, noting that networking control could become that “single point of failure.”
“What now we actually have [is] multiple networks connected to every GPU and they all act independently. Their controls are independent of each other and it's essentially up to the NICs and the GPUs to figure out where to route the packets, and they do that independent of the network controls themselves,” Vincent said. “So that degree of isolation and independence has been very powerful.”
This also allows for more creative software-based network management and maintenance. Vincent provided the example of a customer conducting a large AI training run.
“We really want no entropy in the network,” Vincent noted. So when we do any type of operations, including software operation or physical operations on a network to, let's say clean or fix broken links, we closely coordinate with the customers so that they do critical parts of the training run. We don't actually do any of it. We just let it be and let the software run and really not disturb, and then we have a window where they do something else other than the critical training run. The hardware is still used, but then not for the primary. So they essentially go through phases. Then during that time we essentially go and do a bunch of operations, and that type of model is very atypical for a typical cloud, but that type of model is very successful for large clusters, particularly when they have sensitive training rights.”
Vincent added that this not typically critical for inference runs because of the networking diversity, but it’s big for training and “it's a key differentiator in my opinion in the sense that we have the right infrastructure built in, APIs built in to really coordinate this very closely with key customers.”