{"slug": "the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines", "title": "The P100 has been doing silently noisy math in llama.cpp for years. Three lines fix it for free.", "summary": "A developer discovered that the Tesla P100 GPU has been performing inaccurate floating-point arithmetic in llama.cpp for years due to a missing architecture check. A three-line patch that excludes the P100 from fast fp16 paths reduces median Kullback-Leibler divergence by 2300× and increases top-token agreement from 96.53% to 99.89%, with no performance cost. The fix is now available in community pull requests and an upstream issue has been filed.", "body_md": "**The P100 has been doing silently noisy math in llama.cpp for years. Three lines fix it for free.**\n\nllama.cpp's CUDA backend has a `FAST_FP16_AVAILABLE`\n\nflag: \"this GPU is fast at fp16, so do\nquality-sensitive math in fp16.\" The GTX 10-series (sm_61) was exempted from it long ago. The\nTesla P100 (sm_60) never was — because GP100 is the one Pascal chip with fast fp16 hardware.\nHardware *can*, therefore software *did*. Nobody measured what it cost.\n\nMeasured against an fp32-arithmetic truth base (Qwen3.6-27B Q6_K, wikitext-2, 2048 ctx, 32 chunks, KLD over full logit distributions):\n\n| median KLD vs truth | top-token agreement | |\n|---|---|---|\n| stock llama.cpp on P100 | 0.002298 | 96.53% |\n| 3-line patch | 0.000001 |\n99.89% |\n\nThat's ~2300× tighter, and roughly 1 in 29 next-token predictions were changing outright.\nThe speed cost, benchmarked at pp8192/tg32 @ depth 8192 across three model classes (27B\nhybrid, 4B dense, 36B MoE): **zero**. Prefill inside noise on all three; decode +1.4%\n*faster* patched.\n\nThe patch is merged in [llama-cpp-turboquant PR #212](https://github.com/TheTom/llama-cpp-turboquant/pull/212) and open in\n[buun-llama-cpp PR #80](https://github.com/spiritbuun/buun-llama-cpp/pull/80). GGML upstream issue: [ggml-org/llama.cpp#25593](https://github.com/ggml-org/llama.cpp/issues/25593).\n\n`ggml/src/ggml-cuda/common.cuh`\n\nhas three gates that all say the same thing:\n\n```\n#if defined(FP16_AVAILABLE) && __CUDA_ARCH__ != 610          // device macro\nggml_cuda_highest_compiled_arch(cc) != 610                    // fast_fp16_available()\ncc >= GGML_CUDA_CC_PASCAL && cc != 610                        // fast_fp16_hardware_available()\n```\n\nsm_61 (GTX 1070/1080, P40) is carved out of all three — someone measured or knew that chip's\nfp16 story and made the right call. sm_60 (P100) sailed through all three gates because GP100\nhas genuine 2:1 fp16 hardware (18.7 TF). The fix extends the existing sm_61 idiom:\n`&& __CUDA_ARCH__ != 600`\n\n/ `&& cc != 600`\n\n. Three lines. That's the whole patch.\n\nWhat those gates control on sm_60 (verified in dispatch source, not headers):\n\n**Flash-attention tile and vec kernels**— all of attention math ran in half2 vector arithmetic. (Yes, llama.cpp has real working FA on Pascal — a fact the Python world, where FA2 requires sm_80, mostly doesn't know.)**cuBLAS compute type**— and this one's subtle: on sm_60, MMQ (the int8 quantized matmul path) is*hard-disabled*(`should_use_mmq`\n\nreturns false below the DP4A gate — GP100 missed the DP4A instruction by one minor arch version). So ALL quantized-weight prefill runs dequantize + cuBLAS GEMM, and this flag chose fp16 GEMM. Your entire weight path, in half precision, silently.**mmvf**— f16-weight matrix-vector arithmetic type.\n\nNot by reading code — by measuring GPUs against each other. While validating a KV-cache\ncodec (buun's TCQ/VBR work) across a P100 rig and a 3090, the same model, same weights, same\ntest corpus kept producing systematically different KLD floors. Chasing that difference\nthrough a controlled decomposition panel (KV storage precision × attention arithmetic ×\nweight-path arithmetic, each isolated) pointed at platform *arithmetic*, not the codec.\nThe codec was innocent. The platform wasn't.\n\nKey methodology point for anyone reproducing this class of bug: **within-build comparisons\ncannot see it.** If your truth base and your test run share the same fp16 arithmetic, the\nerror is common-mode and cancels. Every rung of a KV-quant ladder measured on stock sm_60\nlooks clean relative to its own f16 baseline while the whole ladder sits ~0.005 median KLD\naway from the fp32 answer (with a 95.0% same-top floor — the weight-path error is ~2× the\nattention-path error). You must generate the reference logits with fp32 arithmetic to see it.\n\n**sm_60: measured, fixed.** sm_61: was already exempt (nothing changes).**sm_62 (Jetson TX2): same gate, same silicon family, unmeasured — deliberately NOT patched.** Measure before you carve.**Everything Volta+: unaffected by this patch.** Independently verified by the turboquant maintainer pre-merge: these three gates are the only 600-vs-610 distinction in the CUDA tree, and a Blackwell build produced bit-identical PPL. (Whether*other*arches have their own distance-to-fp32-truth is a separate, open research question. Different flag, different kernels, different measurement. Don't extrapolate this bug to your 3090.)\n\nP100s are flooding the used market at ~$80 shipped while the DRAM crisis prices everything else into orbit. That's 16GB of HBM2 at 732 GB/s per card. The market had it half-right: sm_61 P40s (clean math all along) soared to ~$300 while the P100 floundered — the price gap was partly pricing in a software bug. Post-patch, the P100 does fp32-clean inference at fp32 speed that was always there; the \"fast\" fp16 path it loses was buying nothing (real prefill is bound by cuBLAS GEMM and memory bandwidth, not the fp16 vector path).\n\n- Build stock llama.cpp,\n`-DCMAKE_CUDA_ARCHITECTURES=60`\n\n. - Generate truth base:\n`llama-perplexity --kl-divergence-base out.kld`\n\nwith fp32 KV, FA off,**on a patched build**(fp32 arithmetic) — this is the step everyone skips. - Score stock vs patched runs against that base with\n`--kl-divergence`\n\n, 2048 ctx, 32 chunks. - Speed:\n`llama-bench`\n\npp8192/tg32 @ d8192, patched vs unpatched, same host, nothing else on the GPUs.\n\nFull panel design, predictions-logged-before-runs, and receipts: [[https://github.com/apollo-mg/Project-Apollo/blob/1a41c11439e6661c84c83bec333ccdf964663da6/data/Apollo%20Docs/Pascal_FAST_FP16_Carveout_Results.md](https://github.com/apollo-mg/Project-Apollo/blob/1a41c11439e6661c84c83bec333ccdf964663da6/data/Apollo%20Docs/Pascal_FAST_FP16_Carveout_Results.md)]\n[[https://github.com/apollo-mg/Project-Apollo/tree/main/data/carveout_panel](https://github.com/apollo-mg/Project-Apollo/tree/main/data/carveout_panel)].\n\n- buun (TCQ/VBR codec) — the cross-GPU codec validation that surfaced the discrepancy, and the first fork PR review.\n- TheTom (llama-cpp-turboquant) — fast merge with independent cross-arch verification.\n- Found and isolated by running Fable 5 through my custom P/ReAct/R agent loop. It wrote the scripts, the hardware provided the receipts.", "url": "https://wpnews.pro/news/the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines", "canonical_source": "https://gist.github.com/apollo-mg/9218d50a209d70a85f033bf182657818", "published_at": "2026-07-12 05:38:26+00:00", "updated_at": "2026-07-12 20:39:14.006001+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "developer-tools"], "entities": ["llama.cpp", "Tesla P100", "GP100", "Qwen3.6-27B", "GGML", "buun-llama-cpp", "llama-cpp-turboquant"], "alternates": {"html": "https://wpnews.pro/news/the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines", "markdown": "https://wpnews.pro/news/the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines.md", "text": "https://wpnews.pro/news/the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines.txt", "jsonld": "https://wpnews.pro/news/the-p100-has-been-doing-silently-noisy-math-in-llama-cpp-for-years-three-lines.jsonld"}}