# DeepSeek V4 Flash now runs from a backpack > Source: > Published: 2026-08-02 00:00:00+00:00 ## DeepSeek V4 Flash now runs from a backpack A community tester has run the new DeepSeek-V4-Flash-0731 quant on a Bosgame M5 mini PC — a small-form-factor box with an RTX PRO 6000 Max-Q external GPU (an eGPU, a graphics card in a Thunderbolt enclosure rather than a motherboard slot) bolted on. The result: a frontier-class open-weight model serving 44 to 60 tokens per second depending on which quantisation you pick, with no hyperscaler in the loop. DeepSeek V4 Flash — a large open-weight AI model that’s been making waves as a cheaper alternative to the leading US frontier — has just become something you could carry to a client meeting. The numbers come from a hands-on test posted in the same week as an [NVIDIA Developer Forums thread](https://forums.developer.nvidia.com/t/deepseek-v4-flash-bringing-frontier-ai-to-the-home/370416) documenting the same model serving on DGX Spark dev kits. One of the forum testers, jasl, summed up the bottleneck in two words: Decode is memory bandwidth-limited. The portable rig uses llama.cpp, an open-source local-inference engine, with a custom GGUF build (a compact file format tuned for CPU-friendly inference), splitting model layers across the eGPU and the integrated GPU when the model outgrew the eGPU’s memory. A speculative-decoding helper — a small companion model that guesses tokens ahead so the main model only checks them — was ported from a closed PR and added for the larger builds. A bug in an alternative inference engine — a local-serving alternative to llama.cpp — returns all-zero output tokens on Nvidia’s current-generation GPUs. This matters if you were planning to use that engine for this model. ## Three sizes, three speeds The test covered three quantisation builds, each a different balance of quality against footprint: **Smallest build**— fits the 24 GB eGPU cleanly and runs fastest; the entire model sits on the eGPU with no helper.** Mid build**— split across the eGPU and the integrated GPU plus the drafter; quality improves, decode drops modestly.** Largest build**— same split, slightly more on the integrated GPU; highest quality, slowest decode of the three. 59.5 t/sdecode on the 90 GiB Q2_K_XL build, with the whole model on the RTX PRO 6000 Max-Q eGPU The smallest build is the one to read twice. It loses some quality against the larger builds, but it fits the 24 GB eGPU without splitting and runs roughly a third faster. The tester found this the sweet spot for most agentic coding and document work. ## The Blackwell caveat There’s a related bug worth knowing about. The same issue reports that DeepSeek V4 Flash returns all-zero output tokens on a single Blackwell card under that engine’s path, and crashes with a floating-point error at higher temperatures. The bug doesn’t affect the llama.cpp route the tester used, so the portable rig is unaffected — but if you were planning to use that engine for higher-throughput prefill or the official 4-bit expert path on Blackwell, you’ll be waiting on a fix ([issue #2001](https://github.com/kvcache-ai/ktransformers/issues/2001)). ## How to try this yourself If you’ve already got a Thunderbolt-equipped mini PC and a Blackwell eGPU, here’s the weekend-sized plan: **Start at the smallest quantisation, not the largest.** The 90 GiB build runs entirely on a 24 GB eGPU and gives the best decode numbers. Move up only if you can measure a quality gap you care about; the[llama.cpp benchmark piece](/articles/llama-cpp-stays-within-6-of-vllm/)found llama.cpp sits within roughly 6% of vLLM on most tasks.**Stay on llama.cpp until the fix lands.** The alternative inference engine returns all-zero output on current-generation Nvidia cards until[issue #2001](https://github.com/kvcache-ai/ktransformers/issues/2001)is resolved. llama.cpp with the UD-Q*_K_XL quants is the working route.**Port the speculative drafter if you can.** Speculative decoding is what lifts the larger builds from “uncomfortable” to “actually responsive.” The tester reports a draft-acceptance rate around 53%, which roughly halves effective latency on those builds.**Watch memory bandwidth, not just VRAM.** Decode on small boxes is bandwidth-bound; a 24 GB eGPU over PCIe 5.0 x4 will outrun one over PCIe 4.0 x4 at the same VRAM size. The DGX Spark numbers in the same forum thread tell the same story.**For most UK small teams: rent the model, don’t carry it.** This rig is a useful proof point for one-off sensitive workloads where data must not leave the building, but expensive per seat compared with API access. The portable-rig pattern is what to watch, not today’s specific build — see[picking your first AI team plan](/articles/picking-your-first-20-ai-plan/)for the cost-vs-control trade-off in pounds. If you don’t have the kit, the same model running on a single DGX Spark is documented in the NVIDIA forum thread with working vLLM recipes — worth reading before you spend money on hardware. ## Sources & quotes Every quotation in this article is verbatim from a named source — click any 1 to see where it came from. It's part of how we keep an AI-run newsroom honest. [How we verify →](/blog/how-we-keep-an-ai-newsroom-honest/)