{"slug": "qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro", "title": "Qwen3.8-Flash-Next 180B running in real time on a 16G MacBook Pro", "summary": "A 180B-parameter Qwen3.8-Flash-Next model ran fully offline in real time on a 16 GB MacBook Pro using just 9 GB of RAM, with the remaining 112 GB of weights offloaded to the laptop's SSD, according to a post by Jalen Lu on X. Lu said the approach avoids the usual speed penalty of disk offloading by changing when the model knows which weights it needs, though each token still pulls 500 to 700 MB from the SSD, making the drive the bottleneck. Lu argued a frontier model could be designed to be \"prefetchable\" from the start through native training.", "body_md": "Jalen Lu on X: \"This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro!\nQwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’… / X\n\nJalen Lu on X: \"This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro!\nQwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty.\"\n\nThis is a 180B-parameter model running in real time on just a 16 GB MacBook Pro!\nQwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty.\n\nThis is a 180B-parameter model running in real time on just a 16 GB MacBook Pro!\nQwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty.\n\nOffloading a model to disk isn’t new. llama.cpp and others will memory-map a model bigger than your RAM.\nBut the model only finds out which weights it needs at the moment it needs them, so the GPU sits and waits on the drive.\nWe changed when the model knows. Qwen3.8-Flash-NextShow more\n\nEven then, this model was never built to be read off a drive. Each token still pulls 500 to 700 MB from the SSD, so the laptop’s drive sets the speed.\nBut the ceiling is much higher than this! A frontier model can be designed to be “prefetchable” from the start: natively trainedShow more", "url": "https://wpnews.pro/news/qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro", "canonical_source": "https://twitter.com/jalenluorion/status/2122758656668184633", "published_at": "2026-11-17 22:16:15+00:00", "updated_at": "2026-11-17 22:45:38.947571+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-agents", "ai-research"], "entities": ["Qwen3.8-Flash-Next", "Jalen Lu", "OpenCode", "llama.cpp", "MacBook Pro"], "also_reported_by": [], "takeaway": "Offloading 112 GB of a 180B-parameter model to SSD while keeping 9 GB in RAM ran agentic tasks in OpenCode at usable speeds, but each token still pulls 500 to 700 MB from the drive, so SSD bandwidth sets the ceiling.", "recommendation": {"advice": "When serving a model larger than available RAM, offload weights to SSD but schedule prefetch ahead of token generation; the source still measured 500-700 MB of SSD reads per token, so drive bandwidth caps throughput.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro", "markdown": "https://wpnews.pro/news/qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro.md", "text": "https://wpnews.pro/news/qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro.txt", "jsonld": "https://wpnews.pro/news/qwen3-8-flash-next-180b-running-in-real-time-on-a-16g-macbook-pro.jsonld"}}