A September 24, 2026 arXiv paper studying production traces from two companies found that 14 eviction algorithms tested for LLM prefix caching deliver little benefit over LRU despite a large gap to Be…
Modern AI workloads are forcing a fundamental redesign of data center infrastructure, shifting from traditional 5-10 kW racks to high-density facilities constrained by multi-node tensor synchronizatio…
SK hynix showcased its next-generation AI memory portfolio under the slogan 'New Spectrum' at the AI Infra Summit 2026, held September 15th-17th in Santa Clara, California, where the company doubled t…
A technical handbook on FlashAttention explains that the algorithm achieves wall-clock speedups by reducing data movement between GPU high-bandwidth memory (HBM) and on-chip SRAM rather than by approx…
High bandwidth memory (HBM) is hitting scaling limits driven primarily by thermal gradients across the stack, according to a technical deep dive that cites JEDEC standards and Samsung's HBM4 architect…
A forum poster clarified that their original proposal was not to use an NVMe SSD as VRAM, but to design future consumer GPUs with an expandable, lower-cost memory tier managed directly by the GPU, sim…
Kepler Compute, a new memory startup, announced it is developing leadership memory beyond HBM and SRAM that it says can be built without dependence on EUV lithography, with sampling planned for 2026 a…
A user frustrated by GPU memory limitations proposes that consumer graphics cards include an expansion slot for inexpensive NVMe memory to load model caches, arguing this would bypass the high cost of…
SK hynix held its 2026 Future Forum on September 9, 2026, where the company charted its business and technology direction under the theme "Now Is the Golden Time to Leap Forward." The forum covered 3D…
Oxmiq Labs presented high-bandwidth flash (HBF) as a capacity tier for AI inference at Hot Chips 2026, claiming it delivers 8 to 16 times the capacity of HBM at the same cost. The company detailed HBF…
Researchers proposed FLINT, a workload-driven high-bandwidth flash (HBF) substrate that integrates 3D-stacked NAND flash as a memory-capacity tier alongside HBM to accelerate large language model (LLM…
Intel Corporation unveiled its Crescent Island Xe3P inference accelerator at Hot Chips 2026, featuring up to 480 GB of LPDDR5X memory on a single 350 W air-cooled PCIe card, with Intel positioning the…
At Hot Chips 2026, Anurag Agarwal and Radhakrishna Giduthuri presented simulations and projections for High Bandwidth Flash (HBF), a new memory technology that combines SSD-like flash storage with HBM…
RAM prices have surged 500% in 2026, with a 128GB DDR5 kit now costing $3,399, more than 10 times the lowest price ever tracked by Tom's Hardware. The spike is driven by AI infrastructure demand: DRAM…
OVHcloud will raise prices on some dedicated servers in the second half of 2026 due to a global shortage of DDR5 RAM and NVMe drives caused by AI demand, with DDR5 RAM now costing six times more than …
Sandisk taped out its first High Bandwidth Flash (HBF) memory die, a milestone announced at its 2026 Investor Day on August 13, with first HBF inference product samples targeted for 2027. The first-ge…
SRAM serves as the staging ground for KV Cache values written from HBM in LLM inference, and its size directly determines the maximum local data available to local compute, according to a technical de…
South Korea is restructuring its workforce as top students shift from law and medicine to electrical engineering and materials science, driven by the AI chip boom and HBM demand, with pay gaps reversi…
SK Hynix announced it will invest ₩54 trillion ($37.3 billion) to build new DRAM and NAND fabs in Yongin and Cheongju, citing Omdia's projection of 19 percent CAGR for DRAM and NAND demand from 2025 t…
At FMS 2026, SK hynix presented its next-generation memory architecture for the AI era, unveiling the first HBF standard specifications with Sandisk and showcasing AI memory solutions including CXL, D…