I've been looking into the benchmarks for this llama.cpp patch, and the results on an AMD Ryzen AI 9 HX 370 (23.6GB LPDDR5X) are pretty revealing for anyone running LLM agents on consumer hardware:
gpt-oss:120b: Disk 58.5GB → Min RAM 19.1GB (67% savings) at 3.84 tok/sqwen3:30b-a3b: Disk 18.0GB → Min RAM 10.4GB (42% savings) at 19.7 tok/sgemma4:26b-a4b: Disk 16.2GB → Min RAM 10.6GB (35% savings) at 11.5 tok/sGLM-4.7-Flash: Disk 19.0GB → Min RAM 13.3GB (30% savings) at 12.4 tok/s
The most impressive part is that the output is SHA-256 bit-identical to a full-RAM run. This isn't a "compressed" version of the model; it's a lossless deployment.
The technical implementation is surprisingly lean (around 50 lines of C++). It works by profiling MoE expert routing frequencies, using mmap for the whole model, but only using mlock for the "hottest" experts. It then leverages posix_fadvise to prefetch cold experts from the NVMe just before they are needed.
For those of us obsessed with local deployment, this solves a massive pain point. If the model exceeds your RAM, you usually see a performance cliff. HotPin actually provides a 45% speedup in those scenarios (for gpt-oss, it jumped from 2.64 to 3.84 tok/s) compared to standard swapping.
If you want to dive into the implementation, the code is available here:
https://github.com/LozzKappa/hotpin-llm
Next Project Intelligence: Why Git Isn't Enough for AI →