I think my original post may have been misunderstood, so I would like to
clarify what I actually meant.
I was not suggesting that an NVMe SSD should simply be used as VRAM, or
that a GPU should access storage through the normal PCIe → CPU → system
RAM path.
My idea was closer to the concept behind HBM: bringing additional memory
closer to the GPU and designing future consumer GPUs with expandable
memory in mind.
The reason I mentioned cheaper alternatives was because HBM is currently
too expensive for consumer graphics cards. A practical consumer solution
would probably not be identical to HBM, but a new type of GPU memory
expansion layer designed specifically for AI workloads.
The important point is reducing the distance and overhead between the
GPU and additional memory.
A current off approach may look like:
GPU VRAM → PCIe → CPU/system memory → storage
This introduces unnecessary data movement and latency.
What I am suggesting is more like:
GPU → directly managed additional memory tier
without requiring data to travel through the CPU and general-purpose PC
storage path first.
I understand that this would not provide the same bandwidth as HBM or
GDDR VRAM. The goal is not to replace high-speed VRAM, but to create a
more practical memory hierarchy for AI workloads where models are
increasingly limited by memory capacity rather than compute power.
For example, many consumer GPUs today have enough compute capability but are restricted by 8GB, 12GB, or 16GB of VRAM. A better-designed memory
architecture could allow these GPUs to handle larger models with a
reasonable performance tradeoff.
So my suggestion is not “use SSD as VRAM.”
It is: - keep fast VRAM for active workloads, - add a lower-cost expandable memory tier closer to the GPU, - allow the GPU/runtime to
manage data movement efficiently.
The main idea is to rethink consumer GPU memory design for the AI era.