{"slug": "dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and", "title": "Dual Sparks in nvfp4 vs 4x RTX Pro 6000 with native DeepSeek V4 0731 -- Quants and Speed", "summary": "A homelab user deploying DeepSeek-V4-Flash-0731 on two DGX Spark units achieved 1M token context with an NVFP4 KV cache using a custom vLLM fork, reporting a KV cache size of 1,492,347 tokens and maximum concurrency of 1.42x per request. The deployment required a specific configuration including tensor parallelism of 2, dual ConnectX-7 200 Gb/s RoCE fabric, and a custom vLLM 0.21.1rc1.dev339+g1967a5627bc3 runtime, with the model served at 166.9 GB across 48 safetensors shards. The author notes that stock vLLM versions 0.24 and 0.25.2 are incompatible and that deviations from the recommended settings (MAX_NUM_SEQS=6, GPU_MEMORY_UTILIZATION=0.78, and 5 speculative tokens) cause crashes.", "body_md": "DeepSeek-V4-Flash-0731 on 2x DGX Spark: 1M Context, NVFP4 KV Cache\n\nA practical, field-tested companion to the upstream guide:\n\n[GitHub - tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark: DeepSeek V4 Flash DSpark 1M NVFP4 KV recipe for 2x DGX Spark · GitHub](https://github.com/tonyd2wild/DeepSeek-V4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark)\n\nThis document is written for a homelab enthusiast who already owns two DGX Spark units (GB10, SM120), has a ConnectX/QSFP fabric between them, runs Docker, and wants a single 304B-parameter model serving at 1M context with an NVFP4 KV cache. It is a companion, not a replacement. Read the guide first, then use this to avoid the footguns we hit on a real deployment. Everything below was verified on our own pair of Sparks, not copied from the README.\n\n[!IMPORTANT]\n\nThis recipe runs a **custom vLLM fork**, not stock vLLM. Do not point stock > vLLM (main, 0.24, or 0.25.2) at this model on DGX Spark. See > [Why not stock vLLM](#why-not-stock-vllm) below.\n\nThe shape of the thing\n\n| Component |\nValue |\n| Hardware |\n2x DGX Spark (GB10, SM120), 1 GPU per node |\n| Tensor parallelism |\nTP=2 (one GPU per node) |\n| Fabric |\nDual ConnectX-7 200 Gb/s RoCE (RoCEv2, GID index 3) |\n| Base image |\n`ghcr.io/bjk110/vllm-spark:unholy-fusion-prod-ready` (public, no auth, ~21.2 GiB) |\n| Runtime |\nvLLM 0.21.1rc1.dev339+g1967a5627bc3, FlashInfer 0.6.12, TileLang 0.1.9, Python 3.12 at `/opt/env` |\n| Model |\n`deepseek-ai/DeepSeek-V4-Flash-0731` , rev `7872f01b` , 166.9 GB, 48 safetensors shards |\n| KV cache dtype |\n`nvfp4_ds_mla` (the “NVFP4” in the repo name is the KV cache, not the weights) |\n| Context |\n1,048,576 tokens (YaRN factor 16 over 65,536; 1M is the calibrated ceiling) |\n| API |\n`http://<head-fabric-ip>:8888/v1` |\n\nThe build produces four images per node (overlay, stage-a, stage-b, stage-c); **stage-c is canonical** and is what you run.\n\nKey `.env.dspark`\n\nvalues (from our working config)\n\n```\nWORKER_HOST=<worker hostname or IP>\nMASTER_ADDR=10.10.10.1\nMASTER_PORT=25440\nVLLM_HOST_IP=10.10.10.1\nWORKER_VLLM_HOST_IP=10.10.10.2\nNCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1\nNCCL_SOCKET_IFNAME=enp1s0f1np1\nNCCL_IB_GID_INDEX=3\nHF_CACHE=/home/spark2/.cache/huggingface\nDSPARK_MODEL=deepseek-ai/DeepSeek-V4-Flash-0731\n```\n\nOur fabric used `10.10.10.1/.2`\n\n; the guide uses `192.168.192.0/24`\n\n. Your subnet and device names will differ. The values above are the ones that worked for us.\n\nDefault config (current best per the guide, and what we ran):\n\n```\nMAX_MODEL_LEN=1048576\nMAX_NUM_SEQS=6\nGPU_MEMORY_UTILIZATION=0.78\nMTP_NUM_TOKENS=5\nkv-cache-dtype nvfp4_ds_mla\nspeculative config {\"method\":\"dspark\",\"num_speculative_tokens\":5,\"draft_sample_method\":\"probabilistic\"} --distributed-executor-backend mp\n```\n\nDeviating from `MAX_NUM_SEQS=6`\n\n, `GPU_MEMORY_UTILIZATION=0.78`\n\n, or k=5\n\nreproduces documented crashes. The model card recommends\n\n`num_speculative_tokens=7`\n\n, but this drafter’s `dspark_block_size`\n\nis 5 and k=7 does not work on it. Use 5.\n\nLaunch sequence\n\n```\n./build-dspark-vllm-runtime.sh          # build overlay + stage A/B/C on head AND worker ./prepare-dspark-model-cache.sh         # snapshot_download + verify + mirror to worker ./start-deepseek-v4-flash-dspark.sh     # worker-first launch + smoke test\n```\n\nThe API lands at `http://<head-fabric-ip>:8888/v1`\n\n. Boot markers to look for:\n\n```\nGPU KV cache size 1,492,347 tokens Maximum concurrency for 1,048,576 tokens per request: 1.42x Application startup complete.\n```\n\nThen the test: `chat`\n\nwith `\"Reply exactly: NVFP4 DSPARK OK\"`\n\nshould return exactly that string.\n\nWhere the guide needs adjustment (we hit all of these)\n\nThe guide is good, but it assumes a specific lab. On your hardware and setup, you will need to fix these.\n\n1. Baked-in `VLLM_HOST_IP`\n\nfootgun (fatal if ignored)\n\nThe base image has `VLLM_HOST_IP=192.168.192.1`\n\nbaked in. On any node whose fabric IP is not `192.168.192.1`\n\n, the worker dies immediately with:\n\n```\nzmq.error.ZMQError: Cannot assign requested address\n```\n\nOur image also carried a stale `NODE_IP=192.168.50.58`\n\n. Fix: set\n\n`VLLM_HOST_IP`\n\n(head) and `WORKER_VLLM_HOST_IP`\n\n(worker) explicitly per node, and override the stale `NODE_IP`\n\n. Do not rely on anything baked into the image.\n\n2. `NCCL_IB_HCA`\n\nis per-node, not uniform\n\nThe image bakes `NCCL_IB_HCA=rocep1s0f0`\n\n, but RoCE device names are not uniform across units. Auto-detect the Up RoCE device whose netdev carries the fabric IP. On both of our nodes that resolved to:\n\n```\nNCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1\nNCCL_IB_GID_INDEX=3   # RoCEv2\n```\n\nVerify with `ibstat`\n\n/`ibv_devinfo`\n\nrather than trusting the baked value.\n\n3. The model must be on BOTH nodes (TP=2 gotcha)\n\nWith TP=2 every node needs the full 166.9 GB of weights. Download once (to NFS or the head), then rsync to both nodes’ local HF caches. With `HF_HUB_OFFLINE=1`\n\nthe runtime resolves the model fully offline, which is how you avoid a surprise 167 GB re-download at boot.\n\n4. Leftover `DSPARK_MODEL`\n\nstring\n\n`.env.dspark.example`\n\ndefaults `DSPARK_MODEL`\n\nto\n\n`deepseek-ai/DeepSeek-V4-Flash-DSpark`\n\n, which is a leftover name. The real HF repo is `deepseek-ai/DeepSeek-V4-Flash-0731`\n\n. Point it at the 0731 local cache path and run with `HF_HUB_OFFLINE=1`\n\n.\n\n5. `capture_runtime.sh`\n\ncontainer-name mismatch\n\nThe script’s default `CONTAINER_NAME`\n\nis the title-cased basename of the\n\ndirectory, which does not match the lowercase deploy container name. If you do\n\nnot pass it explicitly, the captured files come back empty. Pass:\n\n```\nCONTAINER_NAME=deepseek-v4-flash-0731-dspark-1m-nvfp4-kv-2x-dgx-spark-vllm-dspark-1\n```\n\n6. Reasoning-mode gotcha on the full endpoint\n\nThis only matters if you also run the 4x RTX6000 comparison (see below), but it\n\nis worth knowing: the “full” stock-vLLM deployment runs in REASONING mode. It\n\nemits a separate `reasoning`\n\nfield before `content`\n\n, and small `max_tokens`\n\nbudgets get eaten by reasoning. At `max_tokens=16`\n\nyou get `content: null`\n\n; at 64 you get `\"I am DeepSe\"`\n\n. Use `max_tokens >= 1024`\n\n, or hit the raw\n\n`/v1/completions`\n\npath, which skips reasoning entirely.\n\n7. First boot is slow. Do not panic.\n\nFirst boot takes on the order of 281 s: model load (~79.51 GiB), DeepGEMM warmup, FlashInfer SM120 sparse-MLA autotune, and `torch.compile`\n\n. Later boots reuse caches and are much faster. And before you trust any benchmark number, send real traffic first: cold runs are ~30% low until the JIT kernels warm up.\n\n8. Repo rename\n\nThe repo was renamed from `DeepSeek-v4-Flash-...`\n\nto `DeepSeek-v4-Flash-...`\n\n(a case change). The old `-v4-`\n\nURL still works, but use the current one above.\n\nWhy not stock vLLM\n\nThis is the single most important thing to understand. **Stock vLLM cannot serve this model on DGX Spark.**\n\nThe guide documents this in `UPSTREAM_V024_STATUS.md`\n\n: a 2026-07-01 test of vLLM main (which already contained the merged DSpark PR) imported the DSpark modules but did not boot the checkpoint on 2x DGX Spark/SM120. The failures were backend-level: Unknown SF transformation, a Marlin FP4 repack PTX error, the FlashInfer backend rejecting the device, and MXFP4 method incompatibility.\n\nA direct bake-off in `RUNTIME-BAKEOFF-2026-07-29.md`\n\nagainst vLLM 0.25.2 (`ghcr.io/anemll/dspark-vllm-gx10:0.1.1`\n\n) confirmed the gap: stock lost 9% on peak, 8% on mean, and 29% at concurrency c6. The gap is step time from the B12X MoE kernels plus a working `torch.compile`\n\npath, both of which only exist on the old fork.\n\nThe out-of-tree patch inventory (one line each)\n\n| Patch |\nWhat it does |\nB12X MoE kernels (`b12x_moe.py` , `VLLM_USE_B12X_MOE` ) |\nThe fused MoE kernels that make decode fast on SM120 |\nB12X MLA sparse attention (`b12x_mla_sparse.py` ) |\nSparse-MLA attention for the V4 architecture |\nDeepSeekV4 model support (`models/deepseek_v4/` ) |\nModel implementation for SM120/GB10 |\nSpec-decode proposer Patches 1/2/2b (`dspark_proposer.py` , `dspark.py` , `gpu_model_runner.py` ) |\nThe MTP drafter and its plumbing |\nScheduler Patch 3 (`scheduler.py` ) |\nNo spec placeholders on prefill chunks |\nNVFP4 envelope Stage A/B/C (`nvfp4_ds_mla` dtype) |\n416-byte probe validated to a 584-byte padded envelope |\nPatch 4 (0731, `_STACKED_PARAM_NAME_MAPPING` ) |\nAdds `shared_experts.gate_up_proj` rows; fixes 12 dropped tensors |\n`keys-concurrency.patch` |\nExternal vendored patch |\n`official-main-b12x-nvfp4-python.patch` |\nExperimental, not the default |\n\nPatch 4 is the big one for 0731. It fixes 12 dropped tensors and takes mean\n\ndecode from 32.7 to 55.4 tok/s, peak from 42.0 to 66.1, and acceptance from\n\n25.7% to 60.2%. If your acceptance is stuck in the 20-30% range, Patch 4 is\n\nmissing.\n\nWhere upstream is heading\n\nThe upstream vLLM PRs that this fork tracks or parallels:\n\n`vllm-project/vllm#46995`\n\nSpec Decode DSpark (merged 2026-07-01)\n`vllm-project/vllm#41834`\n\nSM12x support for DeepSeek V4 Flash (open)\n`vllm-project/vllm#43477`\n\nEnable DeepSeek V4 and GLM-5.1 on SM120 (merged)\n`vllm-project/vllm#40082`\n\nFlashInfer B12X MoE and FP4 GEMM for SM120/121 (merged)\n`vllm-project/vllm#43332`\n\nB12X W4A16 (merged)\n`vllm-project/vllm#47577`\n\nAuto-select FLASHINFER_B12X NVFP4 on SM120 (open)\n`vllm-project/vllm#49133`\n\nDSpark draft under its own model/quant config (open; same failure family as Patch 4)\n`vllm-project/vllm#49617`\n\nFix speculators dspark attribute loading (open)\n`vllm-project/vllm#48804`\n\nWarm spec-decode Triton kernels at startup (open)\n`vllm-project/vllm#38476`\n\n/ `#47629`\n\nTRITON_MLA_SPARSE backend (open)\n`vllm-project/vllm#50576`\n\nSM8x support (open issue)\n`vllm-project/vllm#50660`\n\n“0731 cannot be stabilized” (open issue)\n\nRepo issues/PRs on the guide itself (renamed from `-v4-`\n\nto `-v4-`\n\n; the old URL still works): #1 perf PR, #2 long-context CUDA assert, #3 garble-fix root cause, #4 long-context crash fix PR, #5 max-cudagraph truncation PR, #6 open\n\nsoft-failure empty content, #7 proposer fixes via image PR, #8 open engine death CUBLAS/NCCL, #11 multi-node 500 on first request, #12 open “where is the docker image”, #13 open portability PR, #14 open sparkrun recipe PR.\n\nThe takeaway: this is a moving target. The custom fork is the only thing that works today, upstream is converging (several PRs merged), and the open issues above are the ones to watch before you decide to migrate.\n\nModel card facts worth knowing\n\n`deepseek-ai/DeepSeek-V4-Flash-0731`\n\n: 304B params, MIT license,\n\ntext-generation. `max_position_embeddings`\n\nis 1,048,576 with YaRN factor 16 over 65,536 (1M is the calibrated ceiling). Quantization: `expert_dtype fp4`\n\n, weights fp8 e4m3 ue8m0 block 128x128. Config: `num_key_value_heads 1`\n\n, `head_dim 512`\n\n, `q_lora_rank`\n\n/`o_lora_rank 1024`\n\n, `sliding_window 128`\n\n,\n\n`index_topk 512`\n\n, `num_hash_layers 3`\n\n. DSpark is attached; the card recommends `num_speculative_tokens 7`\n\n, but the config has `dspark_block_size 5`\n\nand the guide documents that k=7 does not work on this drafter (use 5).\n\nBenchmark table (measured vs guide claims, steady state)\n\nAll numbers are tok/s (decode and concurrency are aggregate; prefill is TTFT for 1 output token). Measured with `stream:false`\n\n: under spec-decode, streamed deltas measure steps/s, not tokens/s.\n\n| Metric |\nGuide claim |\nMeasured (steady-state) |\nDelta |\n| Decode peak |\n84.3 |\n84.1 |\n-0.2% |\n| Decode mean |\n67.6 |\n67.5 |\n-0.1% |\n| Concurrency c1 |\n61.0 |\n68.8-69.9 |\n+13-15% |\n| Concurrency c2 |\n91.7 |\n104.6-110.7 |\n+14-21% |\n| Concurrency c4 |\n151.1 |\n163.4-169.0 |\n+8-12% |\n| Concurrency c6 |\n197.3 |\n219.2-226.9 |\n+11-15% |\n| Prefill 8K |\n1,513 |\n1,620 |\n+7.1% |\n| Prefill 32K |\n2,284 |\n2,008 |\n-12.1% (single-shot variance) |\n| Prefill 100K |\n2,639 |\n2,690 |\n+1.9% |\n\nBench notes:\n\n`stream:false`\n\nis mandatory. Under spec-decode, vLLM emits one SSE chunk per step, so streamed deltas measure steps/s not tokens/s (we saw 14.7 vs 60.1). Always use `stream:false`\n\nand compute from server `completion_tokens`\n\nover wall time.\n**Heavy warm-up is mandatory.** Cold runs are ~30% low. Our first concurrency pass was depressed by mid-bench Triton/TileLang JIT compilations firing on both ranks (concurrency kernels are not covered by the bench warmup). Once JIT-warm, the re-test met or exceeded every claim. Server logs confirmed zero JIT warnings in the steady-state window.\n- Acceptance measured 62.9% (13,335/21,210) vs the guide’s 60.2% at Patch 4.\n- Zero preemptions, zero queueing, zero OOM across the run.\n\nThe validation thought process\n\nThe goal was to prove the deployment matches the guide, not to re-derive it.\n\nThat shaped the procedure:\n\n**Prove the right code is running.** Patch checks first: `grep -c is_prefill_chunk scheduler.py`\n\nmust return 5, and\n\n`_STACKED_PARAM_NAME_MAPPING`\n\nin `dspark.py`\n\nmust contain the\n\n`shared_experts.gate_up_proj`\n\nrows. If those are missing, every benchmark\n\nbelow is measuring the wrong build. Both passed.\n**Prove the right config booted.** `/v1/models`\n\nmust report\n\n`max_model_len 1048576`\n\n; the log must show the KV cache size, the\n\nconcurrency figure, and `Application startup complete`\n\n.\n**Prove determinism before trusting throughput.** A chat smoke test\n\n(`\"Reply exactly: NVFP4 DSPARK OK\"`\n\n) must return exactly that string. Then\n\nthe agent sanity bench at concurrency 1/2/4/6 with `bad_outputs:0`\n\n.\n**Benchmark decode, then concurrency, then prefill**, each in the guide’s\n\nown terms and each after warm-up, so the comparison is apples-to-apples.\n**Explain the deltas.** Concurrency beats the guide because our runs were\n\nJIT-warm; the 32K prefill dip is single-shot variance, not a regression.\n\nRecord both the first (cold) and steady-state numbers rather than hiding\n\neither.\n\nCommands used to validate\n\n```\n# Boot check\ncurl http://10.10.10.1:8888/v1/models\n# expect max_model_len 1048576; logs show KV cache size + \"Application startup complete\"\n\n# Deterministic sanity\ncurl -s http://10.10.10.1:8888/v1/chat/completions -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"deepseek-v4-flash-dspark\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply exactly: NVFP4 DSPARK OK\"}],\"temperature\":0}'\n# expect: NVFP4 DSPARK OK\n\n# Agent sanity bench (concurrency 1,2,4,6; expect bad_outputs:0)\npython3 scripts/agent_sanity_bench.py   # CONCURRENCY=1,2,4,6\n\n# Full benchmark (heavy warmup mandatory; stream:false)\nURL=http://10.10.10.1:8888/v1 MODEL=deepseek-v4-flash-dspark python3 benchmarks/bench_full.py\n\n# Patch checks\ngrep -c is_prefill_chunk scheduler.py          # expect 5\ngrep -A6 _STACKED_PARAM_NAME_MAPPING dspark.py # expect shared_experts rows\n\n# Runtime capture (pass the container name explicitly!)\nCONTAINER_NAME=deepseek-v4-flash-0731-dspark-1m-nvfp4-kv-2x-dgx-spark-vllm-dspark-1 \\\n  scripts/capture_runtime.sh\n```\n\nValidating this “Spark” model against the native model running on 4x RTX6000s\n\nThis section is about **coherence, not performance**. The RTX6000 box is much faster; that is irrelevant here. The question was narrower: is the model running on 2x DGX Spark through the custom fork the *same model* as the one running natively on 4x RTX6000s through stock vLLM?\n\n**Verdict: yes, it is the same model in two configurations.** 15 of 16 prompts were coherently equivalent (temperature 0, no stream, 16 prompts covering both community and custom tasks). Every deterministic ground-truth task was identically correct on both:\n\n| Task |\nExpected |\nSpark |\nFull |\n| clock_angle |\n7.5 |\n7.5 |\n7.5 |\n| gpqa_balmer |\n656 nm |\n656 |\n656 |\n| gsm8k_eggs |\n18 |\n18 |\n18 |\n| humaneval_close |\n6/6 tests |\n6/6 |\n6/6 |\n| json_format |\nvalid JSON |\nvalid |\nvalid |\n| logic_knights |\nsolve |\nsolve |\nsolve |\n| state_track |\n1 |\n1 |\n1 |\n| structured_json |\nvalid JSON |\nvalid |\nvalid |\n| trap_ignore |\nno banana + ‘4’ |\npass |\npass |\n| trap_no_e |\nzero 'e’s |\npass |\npass |\n| factual_author |\nauthor |\ncorrect |\ncorrect |\n\nThe one caveat with Spark\n\nThere is exactly one divergence, and it is real. `count_letters`\n\nasks “How many times does the letter ‘r’ appear in the word ‘strawberry’?” (correct answer: 3).\n\n**Full (RTX6000):** 3, 3, 3. Stable and correct.\n**Spark (2x DGX):** 5, 5, 9, 5, 9 across five temperature-0 runs. Wrong every time, and non-deterministic at temperature 0.\n\nThis is a genuine, narrow model-quality flake on the custom fork’s sampler, not a configuration artifact. It reproduced in a fresh run. Everything else about the model is coherently equivalent; this one letter-counting edge case is not.\n\nSystematic config differences (not model differences)\n\n**Reasoning field.** Full emits a separate `reasoning`\n\nfield before\n\n`content`\n\n; Spark returns `reasoning: null`\n\n. The final `content`\n\nmatches.\n**Token-budget trap.** At `max_tokens=1024`\n\n, Full’s reasoning ate the budget on a creative task and returned `content: None`\n\n; Spark returned a complete ~150-token continuation. This is why the harness re-runs the creative prompt at 4096 for Full.\n**Verbosity.** Spark is concise (80-170 tokens); Full is expansive\n\n(250-1300 tokens) because it front-loads reasoning. This may be due to slightly differenve vLLM configurations.\n**Hard reasoning** is identical and correct on both, with no truncation on Spark.\n\nReproduction (harness on NFS)\n\nThe harness lives at `/nfs/Dual_Spark_DSV4Flash0731/comparison/`\n\n. It is\n\nstdlib-only, deterministic in its scorers, and was verified reproducible fresh\n\nfrom the NFS copy on stock Python 3.12.\n\n```\ncd /nfs/Dual_Spark_DSV4Flash0731/comparison/\n\n# 1. Edit the two endpoint URLs in compare_harness.py\n#    Spark: http://10.10.10.1:8888/v1   model deepseek-v4-flash-dspark\n#    Full:  http://10.200.0.166:8000/v1 model deepseek-v4-flash\n\n# 2. One-shot collection + scoring + report\nbash run_all.sh out\n```\n\nWhat it does, in one line each:\n\n`compare_harness.py`\n\nsends identical prompts to both endpoints with identical sampling (temperature 0, no stream, generous `max_tokens`\n\n) and saves the raw JSON. It merges runs by prompt+endpoint+max_tokens, keeping the newest, so a creative re-run at a larger budget is picked up automatically.\n`analyze.py`\n\nscores deterministically: numeric exact-match, JSON validity, unit-test pass for the HumanEval-style task, trap checks.\n`report.py`\n\nproduces the per-prompt table plus a full report with quotes and assessments.\n`out/`\n\nholds 32 raw JSON responses plus `records.json`\n\n, `results.json`\n\n,\n\n`comparison_table.md`\n\n, and `results_summary.md`\n\n.\n\nSample run:\n\n```\npython3 compare_harness.py --out out --max-tokens 1024 --timeout 900\npython3 compare_harness.py --out out --max-tokens 4096 --timeout 900 --prompt creative_continuation\npython3 analyze.py --in out --out out\npython3 report.py\n```\n\nFinal checklist\n\n- Fabric IPs assigned and reachable on both nodes;\n`VLLM_HOST_IP`\n\n/\n\n`WORKER_VLLM_HOST_IP`\n\nset explicitly (not the baked-in values).\n-\n`NCCL_IB_HCA`\n\nmatches your RoCE device names; `NCCL_IB_GID_INDEX=3`\n\n.\n- Model cached on\n**both** nodes; `HF_HUB_OFFLINE=1`\n\nfor offline boot.\n-\n`DSPARK_MODEL`\n\npoints at `...-0731`\n\n, not the leftover `-DSpark`\n\nstring.\n- Boot markers present: KV cache size, concurrency figure, “Application\n\nstartup complete”.\n- Smoke test returns exactly\n`NVFP4 DSPARK OK`\n\n.\n- Patch checks pass:\n`is_prefill_chunk`\n\ncount 5, `shared_experts`\n\nrows\n\npresent (Patch 4).\n- Benchmarks run with\n`stream:false`\n\nand after warm-up.\n-\n`capture_runtime.sh`\n\ncalled with the explicit lowercase container name.\n\nThe deployment is real, it matches the guide’s headline numbers (84 tok/s peak, ~67 mean decode, ~220+ aggregate at c6, ~2.7K tok/s prefill at 100K), and the model is the same one you get from stock vLLM on the RTX6000 box, with one documented letter-counting flake on the custom fork’s sampler.\n\n40-minute mixed-traffic soak\n\nAfter the benchmark table above, we ran the guide’s realistic-mixed-traffic soak\n\nto see how the deployment holds up under sustained agent-shaped load. It\n\nreproduced the guide’s numbers within ~2%.\n\n- This run: 532 requests / 209,566 completion tokens / 25,414 prompt tokens /\n\n40 min at concurrency 4, stream:false.\n- Aggregate\n**87.2 tok/s** (guide 88.6, -1.6%), per-stream **21.8** (guide 22.3,\n\n-2.2%). 0 errors, 0 soft-empty, 0 degenerate.\n- Per-stream workers 21.24 / 22.27 / 22.04 / 21.81. Latency wall p50 15.9s,\n\np90 42.1s, p95 44.8s, p99 47.2s, max 52.1s.\n\nMethod\n\nThe guide’s own `benchmarks/soak.py`\n\nis the exact agent-shaped soak behind the\n\nguide’s 553 / 212,974 / 88.6 numbers. We ran an enhanced copy\n\n(`soak_enhanced.py`\n\n) with identical 6 prompt types (tool/code/JSON/reasoning/\n\nlong-context/chat), identical mixed temps (0.0/0.3/0.7) and budgets\n\n(300/500/800), concurrency 4, stream:false, 40 min. We warmed the engine first\n\n(guide’s `bench_full.warm`\n\n: 5 long + 4 short) since warm state decays after\n\n~30 min idle.\n\nServer health\n\nNo OOM, no preemptions, no errors, no restarts on either node; only 1 benign\n\nTriton JIT-compile warning at the soak’s first second; both vllm containers\n\nUp 3h, never restarted.\n\nArtifacts at `/nfs/Dual_Spark_DSV4Flash0731/soak/`\n\n(`soak_enhanced.py`\n\n,\n\n`soak-0731-2xspark.json`\n\n/`.jsonl`\n\n/`.md`\n\n, `soak-calibrate.json`\n\n/`.jsonl`\n\n).\n\nCaveat\n\nThese are mixed-traffic numbers. The same server does 151 agg / 38.7 per-stream on the easy BST benchmark prompt and ~84 peak decode on toy content. ~87/22 is what realistic agent work gets.\n\nExpanded coherence validation: 100 prompts\n\nThe 15/16 result was interesting, so we decided to do another ~100 prompts for more validation.\n\nThe harness is at `/nfs/Dual_Spark_DSV4Flash0731/comparison100/`\n\n(100 prompts, 15 categories, staged and self-tested before the soak, run after the soak finished). The 16 original prompts are included byte-for-byte for continuity, and `counting_traps`\n\nwas deliberately expanded to 13 prompts to stress the `count_letters`\n\nflake.\n\nResult\n\n**83 same / 17 diff / 0 review / 0 fail.** The verdict: the original\n\n`count_letters`\n\nflake was the visible tip of a **SYSTEMIC counting/consistency deficit** on the Spark deployment. Full is correct on all of these.\n\nPer-category rollup:\n\n| Category |\nsame |\ndiff |\n| counting_traps |\n3/13 |\n10 |\n| deterministic_math |\n8/13 |\n5 |\n| logic_puzzles |\n4/5 |\n1 |\n| reasoning_mode |\n3/4 |\n1 |\n| code_generation |\n12/12 |\n0 |\n| code_reasoning |\n5/5 |\n0 |\n| factual |\n12/12 |\n0 |\n| structured_output |\n6/6 |\n0 |\n| traps |\n6/6 |\n0 |\n| translation |\n6/6 |\n0 |\n| summarization |\n4/4 |\n0 |\n| creative |\n4/4 |\n0 |\n| long_reasoning |\n4/4 |\n0 |\n| multiturn |\n3/3 |\n0 |\n| ambiguity |\n3/3 |\n0 |\n\nThe counting traps (Spark wrong, Full correct)\n\n`count_letters`\n\n5 vs 3, `cnt_mississippi`\n\n6 vs 4, `cnt_rhubarb`\n\n5 vs 2,\n\n`cnt_banana_a`\n\n6 vs 3, `cnt_supercali_l`\n\n9 vs 3, `cnt_supercali_i`\n\n9 vs 7,\n\n`cnt_letter_in_sentence`\n\n9 vs 4, `cnt_substr`\n\n6 vs 3, `cnt_vowels`\n\n6 vs 3,\n\n`cnt_triple`\n\n6 vs 3. Spark is only correct on the three that do not require\n\ncounting repeated letters (word-count, string reversal, palindrome).\n\nNon-counting diffs: `math_geom_pyth`\n\n6 vs 5 (hypotenuse 3-4), `math_word_consec`\n\n45 vs 13, `math_word_work`\n\n1.2 vs 2.4, `logic_scheduling`\n\n6 vs 3, `reason_mode_letter`\n\n11 vs 3.\n\nDeterminism (5x temp-0 re-runs)\n\nSpark is non-deterministic on `count_letters`\n\n({9:4, 5:1}) and `math_word_work`\n\n({1.2:4, 2.4:1}); wrong-but-stable on the other wrong counts. Full is deterministic and correct. Two distinct Spark symptoms: (a) a systematic counting/arithmetic-consistency deficit, (b) sampler non-determinism at temp 0 on at least 2 tasks.\n\nArtifacts\n\n`math_prob_dice`\n\n: Spark is correct (writes `\\frac{1}{6}`\n\nin LaTeX); the\n\nfraction scorer’s literal-match check is a false negative.\n`math_word_trains`\n\n: flawed prompt (Train A 60mph leaves earlier AND faster, never catches up); the expected ‘40’ is wrong.\n- Full’s\n`None`\n\non 2 supercali prompts is the 512-token reasoning-budget trap; re-run at 4096 returns the correct 7 and 3.\n\nEverything else is identical: code 17/17 (12 HumanEval all unit tests pass on both, plus 5 code-reasoning), factual 12/12, structured output 6/6, traps 6/6, translation 6/6, summarization 4/4, creative 4/4, long reasoning 4/4 (`train_meet`\n\n10:54 AM solved identically), multiturn 3/3, ambiguity 3/3.\n\nBottom line\n\nSame model, same weights, but the custom vLLM fork on 2x DGX Spark shows a\n\ngenuine, systemic weakness on exact-counting/consistency and trivial arithmetic\n\nthat the stock-vLLM Full deployment does not. The flake was not isolated.\n\nReproduction\n\nHarness at `comparison100/`\n\n(`prompts.py`\n\n/`json`\n\n,\n\n`compare_harness.py`\n\n, `analyze.py`\n\n, `report.py`\n\n, `run_all.sh`\n\n, `README.md`\n\n,\n\n`out/`\n\nwith `records.json`\n\n, `results.json`\n\n, `comparison_table.md`\n\n,\n\n`results_summary.md`\n\n, `raw/`\n\n200 files, `determinism_counting.json`\n\n+\n\n`determinism_math.json`\n\n).\n\nFrom the git clone:\n\n```\ncd comparison100 && ./run_all.sh\n```\n\nProject task: one-page Breakout clone (event-driven JavaScript)\n\nA single 2,500+ token “project” prompt was devised and given to both models.\n\nThe goal was a real, end-to-end software-engineering task rather than short\n\nprompts: build a one-page HTML5 Breakout clone that doubles as a teaching\n\nexample of event-driven JavaScript for a novice. The prompt specified project\n\norganization, code layout, comment style, art/graphics (exact hex palette,\n\n8x6 brick grid), game mechanics, an event-driven teaching angle (CustomEvent\n\npub-sub, event delegation, event.code), a deliberate twist (combo multiplier +\n\nphasing every 5 bricks), Web Audio, localStorage high score, and a strict\n\nquality bar. The full prompt is shared at\n\n`/nfs/Dual_Spark_DSV4Flash0731/project_task/prompt.md`\n\n(~4,480 words).\n\nStats\n\n| Metric |\nSpark (2x DGX) |\nFull (4x RTX6000) |\n| Elapsed |\n229.7 s |\n32.6 s (7x faster) |\n| prompt_tokens |\n5,807 |\n5,807 |\n| completion_tokens |\n13,432 |\n10,303 |\n| Output size |\n67,444 B / 1,551 lines |\n45,779 B / 1,137 lines |\n| finish_reason |\nstop |\nstop |\n| Reasoning field |\nnone |\n1,630 chars |\n| Layout chosen |\nsingle-file index.html |\nsingle-file index.html |\n\nSpec coverage\n\nSpark 33/38 requirements met (5 partial, 0 missing), playable end-to-end.\n\nFull 22/38 (5 partial, 11 missing), NOT playable. Both passed `node --check`\n\n(valid JS) and HTML balance. Review used a headless node vm sandbox with\n\nDOM/canvas stubs to empirically verify runtime behavior, not just static\n\ninspection.\n\nThe twist (combo/phasing)\n\nCORRECT in BOTH artifacts, verified by scripted playthroughs. Combo\n\nincrements, multiplier = min(5, 1 + floor(combo/5)), phasing triggers at\n\ncombo % 5 === 0, resets on miss, 3-second timer expiry, paddle-hit\n\ntermination, lives interaction all per spec. No off-by-one, no wrong reset.\n\nNotable: the Spark deployment’s earlier systemic counting deficit (10/13\n\ncounting prompts wrong) did NOT recur in this code-generation task. Minor\n\nnits: Full dispatches score-update with a stale combo (before increment);\n\nneither implements the spec’s phase countdown meter.\n\nFull model’s critical bugs (why it is not playable)\n\n- F-1 Critical: initAudio() is dead code, never called, so no sound ever plays.\n- F-6 Critical: drawParticles() called without its ctx argument, so render()\n\ncrashes on the first brick break (ctx.globalAlpha on undefined).\n- F-7 High: soft-lock after a miss, no relaunch path.\n- F-2 High: win unreachable, no max level, setState(‘win’) never called.\n- F-3 High: unprotected localStorage.setItem crashes in private mode.\n- F-4 High: two-brick same-frame double-flips the axis, ball tunnels through\n\na brick (the exact bug the spec warned about).\n- F-5 Med: level-complete any-key restarts the game instead of advancing.\n- F-8 Med: no IIFE, global scope pollution.\n\nSpark model’s bugs (all minor)\n\nDead lastHitPaddle field; playSound(‘levelcomplete’) never called; no\n\ncombo-milestone beep; score-update CustomEvent missing bubbles:true; no dedicated corner-stuck detector.\n\nDirect comparison / verdict\n\nSpark produced a more complete, more correct, better-organized artifact (IIFE + single state + setState), with roughly 2x the comment density (404 vs 220 comment lines, all WHY-style teaching comments).\n\nThe 67 KB vs 46 KB size delta is that comment density, i.e. Spark did more of what the spec asked, not padding. Both used the exact 12-hex palette, 8x6 grid, event.code, event delegation, visibilitychange, variable timestep and everything else in the spec document. Teaching sections: Spark 385 words, Full 286, both within the 200-400 word spec.\n\n**Overall verdict**: on a long structured software-engineering task, the 2x Spark deployment with the Dell Pro Max with GB10 (custom vLLM fork) produced the better artifact, slightly, despite Full being 7x faster.\n\nThis inverts the short-prompt pattern: the fork’s counting/consistency deficit appears task-specific (exact counting and arithmetic in short-form), but not a general capability gap. I was surprised by this.\n\nFull’s artifact, while faster to produce, wasn’t runnable without some minor fixes.\n\nUPDATE: There may be a minor problem with vLLM on the Full 4x RTX6000 setup. I am still investigating that part.\n\nArtifacts\n\nTODO\n\nFollow-Up\n\nRequired Reading: [Why your local LLM feels dumber than it is](https://forum.level1techs.com/t/why-your-local-llm-feels-dumber-than-it-is/253917)\n\nfor follow up I’m working on, The rabbit hole on differences beween GH200, sm120 RTX Pro Blackwell and NvFP4 spark has been *interesting* to say the least.", "url": "https://wpnews.pro/news/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and", "canonical_source": "https://forum.level1techs.com/t/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and-speed/253539#post_11", "published_at": "2026-08-22 00:21:03+00:00", "updated_at": "2026-08-22 00:42:46.976315+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["DeepSeek-V4-Flash-0731", "DGX Spark", "vLLM", "NVIDIA", "FlashInfer", "TileLang", "GitHub", "tonyd2wild"], "alternates": {"html": "https://wpnews.pro/news/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and", "markdown": "https://wpnews.pro/news/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and.md", "text": "https://wpnews.pro/news/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and.txt", "jsonld": "https://wpnews.pro/news/dual-sparks-in-nvfp4-vs-4x-rtx-pro-6000-with-native-deepseek-v4-0731-quants-and.jsonld"}}