Clarifying my original idea: I think my intention was misunderstood

A forum poster clarified that their original proposal was not to use an NVMe SSD as VRAM, but to design future consumer GPUs with an expandable, lower-cost memory tier managed directly by the GPU, similar in concept to HBM. The poster argued that current offloading paths — GPU VRAM → PCIe → CPU/system memory → storage — add unnecessary data movement and latency, and that many consumer GPUs with 8GB, 12GB, or 16GB of VRAM are limited by memory capacity rather than compute. The proposal keeps fast VRAM for active workloads while adding a lower-cost tier closer to the GPU so larger AI models can run with a reasonable performance tradeoff.

I think my original post may have been misunderstood, so I would like to clarify what I actually meant. I was not suggesting that an NVMe SSD should simply be used as VRAM, or that a GPU should access storage through the normal PCIe → CPU → system RAM path. My idea was closer to the concept behind HBM: bringing additional memory closer to the GPU and designing future consumer GPUs with expandable memory in mind. The reason I mentioned cheaper alternatives was because HBM is currently too expensive for consumer graphics cards. A practical consumer solution would probably not be identical to HBM, but a new type of GPU memory expansion layer designed specifically for AI workloads. The important point is reducing the distance and overhead between the GPU and additional memory. A current offloading approach may look like: GPU VRAM → PCIe → CPU/system memory → storage This introduces unnecessary data movement and latency. What I am suggesting is more like: GPU → directly managed additional memory tier without requiring data to travel through the CPU and general-purpose PC storage path first. I understand that this would not provide the same bandwidth as HBM or GDDR VRAM. The goal is not to replace high-speed VRAM, but to create a more practical memory hierarchy for AI workloads where models are increasingly limited by memory capacity rather than compute power. For example, many consumer GPUs today have enough compute capability but are restricted by 8GB, 12GB, or 16GB of VRAM. A better-designed memory architecture could allow these GPUs to handle larger models with a reasonable performance tradeoff. So my suggestion is not “use SSD as VRAM.” It is: - keep fast VRAM for active workloads, - add a lower-cost expandable memory tier closer to the GPU, - allow the GPU/runtime to manage data movement efficiently. The main idea is to rethink consumer GPU memory design for the AI era.