Jalen Lu on X: "This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro! Qwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’… / X
Jalen Lu on X: "This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro! Qwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty."
This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro! Qwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty.
This is a 180B-parameter model running in real time on just a 16 GB MacBook Pro! Qwen3.8-Flash-Next, fully offline, using just 9 GB of RAM, doing agentic tasks in OpenCode at usable speeds. The other 112 GB of weights are offloaded to the laptop’s SSD, with a new method that doesn’t pay the usual speed penalty.
Off a model to disk isn’t new. llama.cpp and others will memory-map a model bigger than your RAM. But the model only finds out which weights it needs at the moment it needs them, so the GPU sits and waits on the drive. We changed when the model knows. Qwen3.8-Flash-NextShow more
Even then, this model was never built to be read off a drive. Each token still pulls 500 to 700 MB from the SSD, so the laptop’s drive sets the speed. But the ceiling is much higher than this! A frontier model can be designed to be “prefetchable” from the start: natively trainedShow more