{"slug": "ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc", "title": "SSD-Llama: SSD-Native Inference for Trillion-Parameter Moe on a Consumer PC", "summary": "Researchers submitted SSD-LLaMA to arXiv on 16 Sep 2026, an SSD-native local Mixture-of-Experts inference system that runs trillion-parameter models at over 1 token/s on a single RTX 5090 with no more than 32GB of RAM. Across three frontier MoE model families, SSD-LLaMA improved prefill token rate by 1.52x to 4.19x and decode token rate by 2.10x to 15.58x over evaluated baselines, executing every selected expert without pruning or substitution. The system combines an SSD I/O pipeline optimized for expert delivery, a native three-tier storage hierarchy, and balanced CPU-GPU hybrid execution to keep full expert pools available beyond consumer RAM and VRAM capacity.", "body_md": "# Computer Science > Distributed, Parallel, and Cluster Computing\n\n  [Submitted on 16 Sep 2026]\n\n# Title:SSD-LLaMA: SSD-Native Inference for Trillion-Parameter MoE at 1+ Token/s on a Consumer PC\n\n[View PDF](https://arxiv.org/pdf/2609.18110)\n\n[HTML (experimental)](https://arxiv.org/html/2609.18110v1)\n\nAbstract:Frontier open-weight language models increasingly use Mixture-of-Experts (MoE) architectures to expand model capacity while activating only a small subset of experts per token. Local inference must nevertheless keep the complete expert pool available, which remains far beyond consumer-grade RAM and VRAM capacity even after quantization. SSDs provide practical capacity at this scale, but turning that capacity into executable model memory requires efficient expert delivery, coordinated management of SSD, RAM, and VRAM, and CPU--GPU hybrid execution under bounded bandwidth. We present \\textit{SSD-LLaMA}, an SSD-native local MoE inference system that addresses these challenges with an SSD I/O pipeline optimized for expert delivery, a native three-tier storage hierarchy that delivers and retains experts dynamically, and balanced CPU--GPU hybrid execution. \\textit{SSD-LLaMA} executes every selected expert without pruning or substitution. Across three frontier MoE model families, \\textit{SSD-LLaMA} improves prefill token rate by 1.52$\\times$--4.19$\\times$ and decode token rate by 2.10$\\times$--15.58$\\times$ over the evaluated baselines. We also achieve higher than 1 token/s for running trillion-parameter model with a single RTX 5090 and no more than 32GB RAM.\n    \n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc", "canonical_source": "https://arxiv.org/abs/2609.18110", "published_at": "2026-09-18 18:28:07+00:00", "updated_at": "2026-09-18 18:55:40.039578+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research", "machine-learning"], "entities": ["SSD-LLaMA", "arXiv", "RTX 5090"], "alternates": {"html": "https://wpnews.pro/news/ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc", "markdown": "https://wpnews.pro/news/ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc.md", "text": "https://wpnews.pro/news/ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc.txt", "jsonld": "https://wpnews.pro/news/ssd-llama-ssd-native-inference-for-trillion-parameter-moe-on-a-consumer-pc.jsonld"}}