Marvell, the fabless chip firm labeled the “next trillion-dollar company” by Nvidia CEO Jensen Huang, continued its AI infrastructure march with a solid-state drive (SSD) controller line aimed at boosting storage performance for AI inference workloads.
The Bravera SC6 is a hyperscale-targeted PCIe 6.0 controller designed to smash memory bottlenecks. It acts as a high-capacity, low-cost tiered storage layer to enable key-value (KV) cache – the context history generated during long AI sessions – to move from limited high-bandwidth graphic processing unit (GPU) memory to SSD to improve efficiency.
The vendor claims the device is capable of supporting multigigabyte-per-second throughput and millions of random input/output operations per second (IOPS) to enable rapid access to datasets. By off information that surpasses accelerator memory to its high-capacity non-volatile (NAND) flash memory, Marvell’s Bravera line works to reduce GPU stalls and maintain system utilization.
The card operates at 1.2-volt interfaces across 16 NAND data channels and supports speeds of up to 3,600 megatransfers per second (MT/s). Marvell said that while its Bravera SC6 controller is designed for hyperscale AI and cloud deployments, its “flexible” architecture provides a potential high-end outline for manufacturers to build more enterprise-centric options.
“AI infrastructure is moving beyond isolated servers to systems where compute, memory, and connectivity operate seamlessly together,” Will Chu, EVP and GM of custom cloud solutions at Marvell, explained. “As AI scales, memory must scale more independently of compute so resources can be deployed where they deliver the greatest value.”
AI inference workloads are skyrocketing in both complexity and speeds while memory architectures continue to lag behind. SDxCentral’s Memory & Storage Supplement details that hyperscalers like Meta are re-architecting their storage stacks from the ground up to push past bottlenecks while reducing costs amid supply shortages.
To further help hyperscalers scale AI infrastructure more efficiently, Marvell unveiled a series of photonic fabric technologies. These leverage its optical interconnect work, efforts that Gartner recently said made Marvell the company to beat for AI data center optical connectivity.
A photonic fabric memory module (PFMM) pools and disaggregates memory across AI infrastructure, while a photonic fabric network interface card (NIC) acts as the bridge between host processors and the optical shared-memory fabric.
Marvell said this multirack optical architecture would enable a new shared-memory tier across multiple customized processing units (XPUs) and racks up to 50 meters, enabling up to 32 terabits of warm KV cache offload with high bandwidth and low latency. Such a concept could drastically increase inference throughput and, as a result, boost efficiency of up to three-times higher token throughput within existing data center footprints
The firm also provided a deeper look at its Structera X and A memory controller platforms. At a time when memory hardware costs are spiraling, the vendor wants to help infrastructure operators pool and disaggregate memory resources across servers. Based on the compute express link (CXL) architecture and first showcased some years ago, the latest look at the memory expansion lines sees Marvell’s pledge to optimize resource utilization at the rack-level to help try and bring down total cost of ownership.
“With the industry’s broadest AI memory infrastructure portfolio, Marvell is helping customers improve utilization, boost token efficiency, and scale AI without compromising performance, power, or cost,” Chu added.