SK Hynix looks to be backing optical interconnects and co-packaged optics (CPO) as the technology to break the AI memory wall.
A paper penned by the Korean memory giant along with academic researchers from institutions including the University of Virginia, Nanyang Technological University, and the Massachusetts Institute of Technology examined how optical compute interconnects could support next-generation AI systems.
The researchers back replacing copper interconnects with optical links as “the most promising path toward future scalability.”
CPO technologies are increasingly gaining traction as a means to speed up chip-to-chip links inside compute trays and between switches. But SK Hynix and its research partners view CPO as a means to extend interconnects all the way to the memory interface.
The paper describes an “optics-centric architecture” in which photonic interposers – a chip-packaging substrate that routes data using both electrical signals and light – could be employed to directly connect memory pools and processors in order to maximize the efficiency of data movement across a computing system.
The idea could allow multiple AI accelerators to share large memory pools at vastly higher speeds than conventional storage links, allowing system-level data movement to flow efficiently at scale.
“Extending optical interconnects to the memory interface would overcome the physical constraints surrounding compute chips, removing limitations on both memory capacity and the number of electrical connections,” explained University of Virginia Professor Kyusang Lee. “It would also allow multiple AI accelerators to share a large memory pool, which would make AI infrastructure far more adaptable as AI models continue to grow in scale.”
The ongoing memory shortage, which has resulted in hardware prices skyrocketing, largely stems from AI data center builders. While accelerators have exponentially improved performance levels in recent years, high-bandwidth memory lags as HBM consumes far more silicon wafer capacity per gigabit compared to conventional memory hardware, creating sizable performance bottlenecks. That’s resulted in AI infrastructure operators snapping up all available supply to meet the demands of their high–performance stacks.
Means to smash through the memory wall have seen development of faster, lower-power alternatives like 2T0C from South Korea’s Electronics and Telecommunications Research Institute or chiplet-based interconnects to provide low-latency connectivity between compute engines and memory. Even SK Hynix is throwing its weight behind a potential HBM alternative in the form of the Sandisk-developed high-bandwidth flash.
But given its place as a market leader in HBM, SK Hynix’s backing of CPO acts as a strategic defense mechanism to keep its prized product relevant. After all, the frenzy for HBM propelled the Korean memory giant’s market valuation to eclipse $1 trillion and fuel the largest ever market launch by a foreign firm on the Nasdaq.
“As customers continue to build larger AI systems, the importance of optical interconnects will only increase,” said Seunghoon Hong, AI infra team lead at SK hynix. “Memory companies are evolving beyond the role of supplying individual components to becoming partners that help strengthen the competitiveness of customers’ entire systems through technologies such as CPO.”
“Going forward, we will continue to expand open collaboration with customers and ecosystem partners to accelerate innovation across the AI infrastructure landscape,” Hong added.
More widely, some of the industry’s top interconnect players have joined forces to develop open specifications for CPO in AI infrastructure.
Nvidia, meanwhile, is set to be among the first players to launch CPO network switches in its upcoming Spectrum-6 line – with neocloud darling Lambda providing a sneak peek into the InfiniBand-based switch earlier this year.