cd /news/large-language-models/a-lawyer-spent-9450-on-12-used-nvidi… · home › topics › large-language-models › article
[ARTICLE · art-173965] src=dev.to ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

A lawyer spent $9,450 on 12 used NVIDIA V100 cards to hold every DeepSeek V4.1 Flash expert in GPU memory

A lawyer posting as Nick Mermiges expanded his home AI server from eight to twelve used NVIDIA Tesla V100 32GB GPUs, spending $9,450 to fit all 15,360 routed experts of DeepSeek V4.1 Flash in 384 GiB of HBM2 at 4-bit NVFP4 without further compression. Because the 2018 Volta chips lack FP4 and FP8 tensor cores, the NVFP4 values are unpacked in software to half precision, and Mermiges attributes the speedup to reading fewer bytes rather than faster compute. He self-reports up to 184 tokens per second on single-stream code generation and roughly half the time to first token on long documents versus the prior eight-card build, though gains vanish at 32K context and above.

recommendation

When serving MoE models on memory-bound older GPUs, keep experts at 4-bit and unpack in software rather than compressing further; the builder halved first-token time at 256K context by fitting all experts in HBM.

more in playbook: Inference and self-hosting
by read3 min views6 publishedNov 21, 2026

📅 November 20, 2026 • Tom's Hardware

A lawyer who posts on r/LocalLLM as Nick Mermiges has expanded his home AI server from eight to twelve used NVIDIA Tesla V100 32GB cards, adding a third NVLink board of four GPUs. Per his post and GitHub write-up, covered by Tom's Hardware on November 20, 2026, the whole build cost $9,450. The machine now runs DeepSeek V4.1 Flash with all 15,360 routed experts resident in memory at 4-bit NVFP4, and his repo notes none of them were compressed further.

The memory math explains the third board. Eight 32GB cards offered about 254 GiB, while the NVFP4 experts alone need 278.9 GiB across 40 layers, per the builder's notes. Twelve cards lift total HBM2 memory to 384 GiB, so the full expert set fits without further shrinking. Before, he ran every expert at 2-bit precision and moved only the 5,949 most-used ones, about 38.7 percent, to NVFP4 until the slots filled.

There is a catch in where the speed comes from. The V100 dates to 2018, and its Volta chips have no FP4 or FP8 tensor cores, so every NVFP4 number is unpacked in software into half precision before use. Per Mermiges, the benefit appears because the cards read fewer bytes, not because they compute faster, and he writes that expert values equal the ones in DeepSeek's own release while other parts of the model use compact forms.

All performance figures here are self-reported by the builder. Per Mermiges, code generation reaches up to 184 tokens per second on a single stream, and long documents start answering in about half the time versus the previous eight-card build, which he calls the main improvement. He reports the first code token at 256K context in 58 seconds versus 108.4 seconds before, and prompt reading above 32K context at about 4,500 tokens per second versus 2,200 on the first build.

The limits are stated honestly. Code decoding is 23 percent faster at 1K context per his write-up, but he sees no real gains at 32K context and above. Reading a 128K file takes 28 seconds and a 256K file takes 185 seconds, and three parallel requests at 128K yield about 360 tokens per second on code or 213 on legal documents, per Mermiges.

He also says the machine beats published four-unit DGX Spark stacks at single-request decoding, by 15 to 45 percent on code, at about half the price of a three-unit Spark setup. Tuning helped too. He raised the power cap from 150 to 175 watts and lowered the clock cap from 1,522 to 1,452 MHz for a net performance gain, per his notes.

Why it matters. Discontinued 2018 silicon is running a 2026 model format through software workarounds, and the project shows budget local AI is often bound by memory bandwidth rather than compute. It remains a hobbyist machine with self-reported numbers, not an independent benchmark, and per his documentation the third board sits behind a separate PCIe switch whose second hop adds latency. For a lawyer reading long contracts locally, halving the wait for the first token may be worth every dollar of the $9,450.

Sources:

https://tomshardware.com/tech-industry/artificial-intelligence/lawyers-usd9-450-private-ai-server-of-12-used-nvidia-v100s-now-runs-every-deepseek-v4-1-flash-expert-at-4-bit-nvfp4-up-to-184-tokens-per-second-on-code-and-long-documents-start-answering-in-about-half-the-time

https://mezha.net/eng/news/mezha-268801830767cbc0b0dce67c_lawyer-s_-9-450_server https://ameriledger.com/article/lawyer-s-9-450-private-ai-server-with-12-used-nvidia-v100s-now-runs-every

[https://urgent.news/2026/11/20/lawyers-9-450-private-ai-server-with-12-used-nvidia-v100s-now-runs](https://urgent.news/2026/11/20/lawyers-9-450-private-ai-server-with-12-used-nvidia-v100s-now-runs)

[https://technetbooks.com/2026/11/lawyer-builds-personal-ai-cluster-to.html](https://technetbooks.com/2026/11/lawyer-builds-personal-ai-cluster-to.html)

https://gate.com/news/detail/lawyer-upgrades-private-ai-server-to-12-nvidia-v100-gpus-for-deepseek-v41-25752904

Sources:

Continue the conversation: https://t.me/AIAutomationArchitecture · Bluesky

── more in #large-language-models 4 stories · sorted by recency
── more on @nvidia 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/a-lawyer-spent-9450-…] indexed:0 read:3min 2026-11-21 · —