A homelabber has pushed Moonshot AI’s 2.8-trillion-parameter Kimi K3 to 3.11 tokens per second on a discontinued Optane PMem server. The trick was not the two RTX 3090s he added, but moving nearly half the model off Optane and into plain DRAM.
Two weeks ago, Reddit user IndividualSurvival showed up on r/homelab with a slightly mad project: running Kimi K3, Moonshot AI’s 2.8-trillion-parameter mixture-of-experts model, on a 40-core third-gen Xeon Scalable box stuffed with Intel Optane PMem 200 modules. No GPU at all. It worked, in the sense that it produced text, at 0.85 tokens per second with a 1-bit quant. Turning off DAX and letting the OS page cache do its thing later lifted that to 1.50 tokens per second.
The follow-up, reported by Tom’s Hardware and TweakTown, is a lot more usable. With a Q2_K quant, two GeForce RTX 3090s and a RAM drive, the same server now generates 3.11 tokens per second.
Where the speed actually came from #
The model file is about 865GB, split into 94 shards and loaded through llama.cpp in a hybrid CPU-GPU setup. The two 3090s, 48GB of VRAM between them, hold only around 5.5% of that file. They take the attention layers, the dense feed-forward blocks, the shared experts and the KV cache. The routed experts, which make up the bulk of the weights, stay in system memory.
With all 94 shards sitting on Optane and the GPUs already installed, the build managed 2.20 tokens per second for generation and 13.3 tokens per second for prompt processing. Then the builder copied 45 of the shards, roughly 414GB, into a tmpfs RAM drive and left the other 49 on Optane. Generation rose to 3.11 tokens per second and prompt processing to 16.8. That is about 41% and 26% faster, respectively, with the exact same pair of graphics cards.
So the GPUs make the whole thing viable, but the jump in the latest round is a memory story. Optane PMem was always pitched as a middle tier, denser and cheaper than DRAM but slower, and a mixture-of-experts model that pulls a different set of experts for every token hits that latency gap constantly. Kimi K3 activates around 104 billion parameters per token, so every fraction of the expert pool that moves into DRAM shaves time off each step.
Heat, metal cutting and a reality check #
None of this was plug and play. Fitting two 3090s meant cutting into the case and relocating the power supply. The Optane DIMMs are their own problem: Intel rates PMem 200 modules at up to 18W each, with throttling at 83°C and shutdown at 85°C. The builder runs two 12W blower fans aimed straight at the memory and calls active DIMM cooling mandatory, warning that a hot Optane stick can cook the DDR4 sitting next to it.
And 3.11 tokens per second is still slow. It is below a comfortable reading pace, and a long answer will take minutes rather than seconds. The figures are also self-reported and run on a single configuration, so treat them as one builder’s numbers, not a benchmark.
Still, the experiment says something useful. PMem 200 is discontinued, which is exactly why second-hand modules are showing up in homelabs at prices DRAM cannot match per gigabyte. For people determined to run trillion-parameter open-weight models locally, the recipe here is clear enough: put the dense parts on whatever GPUs you have, get as much of the expert pool into real DRAM as you can afford, and let Optane catch the rest. Just point a fan at it.