{"slug": "768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps", "title": "768GB Intel Optane DIMMs to run 1T-parameter LLM with single GPU at 4tps", "summary": "A Redditor running a workstation with 768GB of used Intel Optane Persistent Memory DIMMs achieved roughly 4 tokens per second on a 1-trillion-parameter LLM (Kimi K2.5) using a single GPU. The six second-hand Optane sticks, acquired cheaply, offer lower latency than NVMe SSDs but remain slower than DRAM, enabling local inference of massive models at a fraction of the cost of equivalent DRAM. The feat highlights the viability of discontinued Optane memory for large-scale AI workloads, though its scarcity limits the approach to an exotic, non-scalable solution.", "body_md": "768GB of cheap Intel Optane DIMM memory sticks used to run 1-trillion-parameter LLM on a system with a single GPU — local Kimi K2.5 install achieved roughly 4 tokens per second\n\nA Redditor has caused a stir by coaxing a workstation build using Optane PMem DIMMs as RAM to run a 1-trillion-parameter LLM. APFrisco explains in a mini tutorial/guide on the Local LLaMA subreddit how they bought some used Intel Optane Persistent Memory, acquired relatively cheaply second-hand, to “run a 1 trillion parameter model (in this case Kimi K2.5) locally at ~4 tokens/second” on their Xeon workstation.\n\nCentral to the headlining feat was the Redditor’s sourcing of six Optane PMem (DCPMM) sticks. The discontinued memory format was designed to bridge the DRAM-SSD divide. While the 768GB of Optane (6x 128GB) does indeed offer far lower latency than the best NVMe SSDs, it is still two or three times slower than DRAM. These characteristics are still rather sweet for LLM inference frameworks, and the second-hand price was “much less than what the equivalent DRAM capacity would cost.” But, alas, Optane is dead, so this is an exotic solution.", "url": "https://wpnews.pro/news/768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps", "canonical_source": "https://www.tomshardware.com/tech-industry/artificial-intelligence/enthusiast-runs-1-trillion-parameter-llm-from-768gb-of-intel-optane-dimm-memory-sticks-local-kimi-k2-5-install-achieved-roughly-4-tokens-per-second", "published_at": "2026-05-30 20:17:49+00:00", "updated_at": "2026-05-30 20:47:22.521336+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-infrastructure", "ai-chips"], "entities": ["Intel", "Optane", "Kimi K2.5", "APFrisco", "Local LLaMA", "Reddit"], "alternates": {"html": "https://wpnews.pro/news/768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps", "markdown": "https://wpnews.pro/news/768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps.md", "text": "https://wpnews.pro/news/768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps.txt", "jsonld": "https://wpnews.pro/news/768gb-intel-optane-dimms-to-run-1t-parameter-llm-with-single-gpu-at-4tps.jsonld"}}