Kioxia’s Flash-for-DRAM Initiative Eyes AI Workloads

Kioxia presented a Compute Express Link (CXL) module at a Tokyo briefing that uses its XL-Flash NAND to replace some DRAM in AI workloads, claiming it can double memory capacity at the same cost while improving performance by 30%. The Japanese memory chipmaker, which lacks a DRAM portfolio unlike rivals Micron, Samsung, and SK Hynix, aims to offset competitive pressures from China's Yangtze Memory Technologies Co. (YMTC) by targeting data center tiering and edge AI applications.

Kioxia has moved one step closer to its ambitious goal of replacing DRAM with flash for some memory content serving AI workloads. It presented a Compute Express Link CXL module at a briefing in Tokyo; the module is built around XL-Flash, a specialized NAND chip with sharply improved processing speeds. This CXL module—aimed at replacing part of DRAM content alongside AI compute—uses NAND flash optimized for high-speed processing. Kioxia claims that replacing some DRAM with CXL modules can double memory capacity at the same cost while improving performance by 30%. CXL https://www.eetimes.com/stripped-down-cxl-scales-memory-wall/ —an interface standard that ties together different memory types and devices inside a system into a single large memory space—allows capacity to be allocated dynamically as needed. As a result, a NAND-based CXL module is cost-competitive against DRAM. While DRAM reads data extremely fast, it is difficult to expand in capacity and is more expensive. On the other hand, cheaper NAND-based CXL enables an ultra-large memory pool https://www.eetimes.com/from-gpus-to-memory-pools-why-ai-needs-compute-express-link-cxl/ at a lower cost than a DRAM-only configuration. Here, substituting NAND for DRAM in certain AI workloads hinges on evolving system architectures that can tolerate higher latency in exchange for greater density and lower cost per bit. View All https://www.eetimes.com/category/sponsored-content/ Flash window of opportunity NAND flash offers substantially more storage capacity at a given price point, but its access speeds are measured in microseconds rather than DRAM’s nanoseconds. However, emerging applications in data center tiering and edge AI now provide new openings for systems designed to manage mixed-latency storage hierarchies. These systems can route less time-sensitive operations to flash-based memory while reserving DRAM for performance-critical tasks. Consider machine learning ML model inference in data center environments, where latency may not matter much unless it’s more than an order of magnitude worse than high-bandwidth memory HBM . Next, in edge AI, which commonly employs pre-trained models, Kioxia’s CXL module aligns with NAND’s advantage of large capacity. It’s also immune to NAND’s weaknesses of slow writes and limited endurance, and its low-power consumption suits energy-sensitive edge environments. At this memory-technology crossroads, a CXL module can boost NAND flash memory processing speeds as part of the initiative to replace some DRAM demand in AI servers. Consequently, if NAND can absorb some DRAM capacity in data center servers and emerging applications such as edge AI, the addressable market for flash widens at the expense of DRAM chips. Nevertheless, Kioxia’s flash-for-DRAM initiative is a supplementary strategy rather than a core design initiative. It heeds specialized NAND applications that can offset the absence of DRAM. But the question is: why Kioxia https://www.eetimes.com/kioxia-all-set-to-raise-the-nand-game-in-ai-ssds/ ? Kioxia’s memory imperative Unlike other top-tier memory producers—Micron, Samsung, and SK Hynix—Kioxia lacks a DRAM portfolio. That leaves the Japanese memory chipmaker exposed in a market where its key rivals can leverage both DRAM and flash memory technologies. In other words, Kioxia’s singular focus on NAND flash has become a liability. Moreover, within the NAND flash space, China’s Yangtze Memory Technologies Co. YMTC https://www.eetimes.com/chinas-nand-specialist-ymtc-moves-closer-to-ipo/ is narrowing the multi-year technology gap in a matter of months. That especially threatens Kioxia’s position in price-sensitive memory segments. Then there is Micron, which has regained momentum in the NAND flash market after a difficult stretch spanning many years. Such competitive pressures have left Kioxia searching for strategic differentiation. Here, it must extract value from its existing NAND capabilities through novel initiatives: enter the NAND-based CXL module to replace some DRAM demand. But that’s only half of the story. In NAND-based CXL module design, while technical tradeoffs are well understood, implementation requires software stacks and system designs that exploit the cost-density advantages while managing latency penalties. That, in turn, calls for ecosystem support from server manufacturers, hyperscalers, and software vendors. Without design wins and ecosystem support, it will remain a forward-looking statement regarding memory architecture rather than a memory industry breakthrough. Therefore, industry watchers are keenly awaiting design win announcements from Kioxia; they will validate the technical premise of NAND substitution in memory-intensive AI workloads. Until then, Kioxia carving out a middle path to position flash as a tier between DRAM and storage will remain a work in progress. While the technical details of this unlaunched module are making waves in the memory world, flash as a DRAM proxy must open a differentiated revenue stream to become a persistent HBM twin. See also: Stripped Down CXL Scales Memory Wall https://www.eetimes.com/stripped-down-cxl-scales-memory-wall/ NAND Flash’s Reversal of Fortune Amid the AI Boom https://www.eetimes.com/nand-flashs-reversal-of-fortune-amid-the-ai-boom/ From GPUs to Memory Pools: Why AI Needs Compute Express Link CXL https://www.eetimes.com/from-gpus-to-memory-pools-why-ai-needs-compute-express-link-cxl/