DeepSeek v4.1 flash runs 23 seconds/token on a 2020 16gb M1 Mac Mini

A developer posting as FP4 Brain on X reported running DeepSeek V4.1 Flash locally on a 16GB M1 Mac Mini using original FP4/FP8 weights, SSD streaming, and a custom MLX runner built on the pipenetwork MLX port, achieving 108 seconds time-to-first-token and about 23 seconds per token. The developer said the setup improved from roughly 31 to 23 seconds per token by reusing allocation buffers, compiling weight decoding, and caching 4 GiB of dense weights, keeping weights on the SSD with selected experts and lookup rows loaded on demand; a 6 GiB cache was tried but was barely faster with more swapping. Code, tests, and logs are published in the GitHub repository atbender/deeps.

FP4 Brain on X: "got deepseek V4.1 flash running locally on a 16GB m1 mac mini original FP4/FP8 weights, ssd streaming + custom mlx runner 108s ttft and about 23s/token not to be confused with tok/s " got deepseek V4.1 flash running locally on a 16GB m1 mac mini original FP4/FP8 weights, ssd streaming + custom mlx runner 108s ttft and about 23s/token not to be confused with tok/s got deepseek V4.1 flash running locally on a 16GB m1 mac mini original FP4/FP8 weights, ssd streaming + custom mlx runner 108s ttft and about 23s/token not to be confused with tok/s code + recipe if you're not in a hurry github.com/atbender/deeps… weights stay on SSD. the runner loads selected experts + lookup rows on demand, with a 4 GiB cache for dense weights. built on @pipenetwork MLX port got it from about 31 to 23 seconds/token by reusing allocation buffers, compiling weight decoding and caching 4 GiB of dense weights tried 6 GiB too but it was meh, barely faster and more swapping. that's why I stuck with 4 if anyone's wondering tests + logs in the repo