Ggml v1.0.0

Ggml released version 1.0.0, pruning its deprecated public API and removing the legacy DIAG_MASK_INF and DIAG_MASK_ZERO ops while bumping the RPC protocol to v9 to reject incompatible peers. The release adds configurable YaRN truncation via ggml_rope_set_truncate and ggml_rope_get_truncate, OCP FP8 support on CUDA, and backend performance work including few-row MMQ for batch sizes 2-24, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, and scheduler changes that skip uploading inputs that are not read. The API pruning and RPC protocol bump mean existing ggml-based integrations must update to remain compatible with peers running 1.0.0.

Repository navigation Overview ggml 1.0.0 prunes the deprecated public API and the legacy DIAG MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quantized matmuls and MoE fusion on CUDA/Metal, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, new WebGPU kernels, expanded Hexagon op coverage, and scheduler improvements that avoid uploading unused inputs. API changes - Pruned deprecated API and removed the legacy DIAG MASK INF / DIAG MASK ZERO ops, bumping the RPC protocol to reject incompatible peers llama/29801 https://github.com/ggml-org/llama.cpp/pull/29801 , llama/31946 https://github.com/ggml-org/llama.cpp/pull/31946 - Added ggml rope set truncate / ggml rope get truncate to control YaRN correction-bound rounding llama/30206 https://github.com/ggml-org/llama.cpp/pull/30206 , llama/30802 https://github.com/ggml-org/llama.cpp/pull/30802 - Implemented the backward pass for TRI llama/30712 https://github.com/ggml-org/llama.cpp/pull/30712 - Added MUL MAT src1 precision tests and gating for ggml prec set acc llama/31896 https://github.com/ggml-org/llama.cpp/pull/31896 , llama/31851 https://github.com/ggml-org/llama.cpp/pull/31851 - Bumped the RPC protocol to v9 with faster loading and graceful failure on unreachable endpoints or failed handshakes llama/29413 https://github.com/ggml-org/llama.cpp/pull/29413 , llama/30415 https://github.com/ggml-org/llama.cpp/pull/30415 Core changes - Scheduler skips uploading inputs that are not read and prefetches only inputs produced by earlier splits llama/31419 https://github.com/ggml-org/llama.cpp/pull/31419 , llama/32100 https://github.com/ggml-org/llama.cpp/pull/32100 - MoE cache can compute a share of the cache misses on the CPU and copy the results back asynchronously llama/30966 https://github.com/ggml-org/llama.cpp/pull/30966 - Meta backend raises the compute view limit for buffers with few tensors llama/31447 https://github.com/ggml-org/llama.cpp/pull/31447 Backend changes CUDA - Added few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls llama/31533 https://github.com/ggml-org/llama.cpp/pull/31533 , llama/31959 https://github.com/ggml-org/llama.cpp/pull/31959 - Added OCP FP8 support and fused NVFP4 gate/up matmuls with GLU in MMQ llama/28413 https://github.com/ggml-org/llama.cpp/pull/28413 , llama/31296 https://github.com/ggml-org/llama.cpp/pull/31296 - Extended FlashAttention with an sm70 head-size-256 config and MMA head sizes 40/72 llama/27997 https://github.com/ggml-org/llama.cpp/pull/27997 , llama/30285 https://github.com/ggml-org/llama.cpp/pull/30285 - Improved graph handling: skip compat checks on reuse, re-instantiate on cluster-dim changes, move graph state llama/32004 https://github.com/ggml-org/llama.cpp/pull/32004 , llama/31284 https://github.com/ggml-org/llama.cpp/pull/31284 , llama/32001 https://github.com/ggml-org/llama.cpp/pull/32001 - MoE: skip empty MMQ stream-k tiles, linearize the MoE layout, and sanitize post-bias NaNs llama/32067 https://github.com/ggml-org/llama.cpp/pull/32067 , llama/31047 https://github.com/ggml-org/llama.cpp/pull/31047 , llama/31005 https://github.com/ggml-org/llama.cpp/pull/31005 - Added a debug buffer pool and halved the VMM pool reservation with reserve retries llama/31919 https://github.com/ggml-org/llama.cpp/pull/31919 , llama/30542 https://github.com/ggml-org/llama.cpp/pull/30542 - HIP: use a fixed BLAS workspace and add an MFMA lightning indexer for CDNA2 llama/31838 https://github.com/ggml-org/llama.cpp/pull/31838 , llama/29050 https://github.com/ggml-org/llama.cpp/pull/29050 - Added a d state=64 SSM SCAN fast path and made radix Top-K deterministic for ties llama/31818 https://github.com/ggml-org/llama.cpp/pull/31818 , llama/31084 https://github.com/ggml-org/llama.cpp/pull/31084 Metal - Fused MoE routing and kept fusable node groups inside a single command buffer llama/31615 https://github.com/ggml-org/llama.cpp/pull/31615 , llama/31688 https://github.com/ggml-org/llama.cpp/pull/31688 - Made radix Top-K deterministic for ties, added mul mm for 32<=K<64, and switched TRI to select llama/31084 https://github.com/ggml-org/llama.cpp/pull/31084 , llama/30982 https://github.com/ggml-org/llama.cpp/pull/30982 , llama/30711 https://github.com/ggml-org/llama.cpp/pull/30711 - Fixed GROUP NORM for ne03 1 and out-of-bounds accesses in the FA output and SOLVE TRI kernels llama/31852 https://github.com/ggml-org/llama.cpp/pull/31852 , llama/31662 https://github.com/ggml-org/llama.cpp/pull/31662 , llama/32103 https://github.com/ggml-org/llama.cpp/pull/32103 Vulkan - Added an Intel FlashAttention prefill kernel llama/29357 https://github.com/ggml-org/llama.cpp/pull/29357 - Added batched tensor copies, a test-fusion interface, and ReBAR detection llama/31418 https://github.com/ggml-org/llama.cpp/pull/31418 , llama/30769 https://github.com/ggml-org/llama.cpp/pull/30769 , llama/31658 https://github.com/ggml-org/llama.cpp/pull/31658 - Enabled dequantized KV with CM2, YaRN truncation, RDNA3/4 large-tile split-K, and large coopmat matmul on newer AMD drivers llama/31526 https://github.com/ggml-org/llama.cpp/pull/31526 , llama/30802 https://github.com/ggml-org/llama.cpp/pull/30802 , llama/30280 https://github.com/ggml-org/llama.cpp/pull/30280 , llama/31587 https://github.com/ggml-org/llama.cpp/pull/31587 - Added a tiled-transpose concat shader and FWHT kernels for block widths above 512 llama/30591 https://github.com/ggml-org/llama.cpp/pull/30591 , llama/29772 https://github.com/ggml-org/llama.cpp/pull/29772 - Fixed MoltenVK out-of-bounds accesses, GROUP NORM workgroup limits, and FA scratch sizing llama/31890 https://github.com/ggml-org/llama.cpp/pull/31890 , llama/31930 https://github.com/ggml-org/llama.cpp/pull/31930 , llama/30487 https://github.com/ggml-org/llama.cpp/pull/30487 - Capped scalar FlashAttention at 8 rows for large heads on Qualcomm llama/30829 https://github.com/ggml-org/llama.cpp/pull/30829 SYCL - Added a device-side MoE scheduler llama/30156 https://github.com/ggml-org/llama.cpp/pull/30156 - Made the oneDNN FA cache thread-safe and un-gated fused SDPA for Alchemist at head dim=64 llama/31138 https://github.com/ggml-org/llama.cpp/pull/31138 , llama/31574 https://github.com/ggml-org/llama.cpp/pull/31574 - Cached the device id and reject buffers that would exceed VRAM llama/31126 https://github.com/ggml-org/llama.cpp/pull/31126 , llama/32013 https://github.com/ggml-org/llama.cpp/pull/32013 , llama/31285 https://github.com/ggml-org/llama.cpp/pull/31285 - Added exact rounding for CPY to Q4 1/Q5 1/Q8 0, strided CLAMP, and f16 SWIGLU OAI llama/31465 https://github.com/ggml-org/llama.cpp/pull/31465 , llama/31784 https://github.com/ggml-org/llama.cpp/pull/31784 , llama/30857 https://github.com/ggml-org/llama.cpp/pull/30857 - Fixed AOT for allocations above 4GB llama/31873 https://github.com/ggml-org/llama.cpp/pull/31873 OpenCL - Added decomposed Adreno FlashAttention and improved decode performance llama/32046 https://github.com/ggml-org/llama.cpp/pull/32046 , llama/31633 https://github.com/ggml-org/llama.cpp/pull/31633 - Added q6 k tiled, q8 0, and f32/q5 k multi-column GEMV kernels and f32 MUL MAT ID llama/31758 https://github.com/ggml-org/llama.cpp/pull/31758 , llama/30947 https://github.com/ggml-org/llama.cpp/pull/30947 , llama/31783 https://github.com/ggml-org/llama.cpp/pull/31783 - Added arange, non-contiguous sigmoid, repeat for f16/i32/i16/bf16, and DSV4 HC pre/post kernels llama/31829 https://github.com/ggml-org/llama.cpp/pull/31829 , llama/31687 https://github.com/ggml-org/llama.cpp/pull/31687 , llama/31782 https://github.com/ggml-org/llama.cpp/pull/31782 , llama/31623 https://github.com/ggml-org/llama.cpp/pull/31623 - Enabled multi-column GEMV by default for Adreno X2 llama/31694 https://github.com/ggml-org/llama.cpp/pull/31694 WebGPU - Added batched tensor API support and returned maxBufferSize from get max size llama/31473 https://github.com/ggml-org/llama.cpp/pull/31473 , llama/31348 https://github.com/ggml-org/llama.cpp/pull/31348 - Added GROUP NORM, MEAN, COUNT EQUAL, and PAD REFLECT 1D kernels llama/31490 https://github.com/ggml-org/llama.cpp/pull/31490 , llama/31591 https://github.com/ggml-org/llama.cpp/pull/31591 , llama/31578 https://github.com/ggml-org/llama.cpp/pull/31578 , llama/31877 https://github.com/ggml-org/llama.cpp/pull/31877 - Supported the YaRN truncation config llama/30802 https://github.com/ggml-org/llama.cpp/pull/30802 Hexagon - Added Q5 0/Q5 1 support and optimized conv 2d while adding conv 3d llama/32094 https://github.com/ggml-org/llama.cpp/pull/32094 , llama/31814 https://github.com/ggml-org/llama.cpp/pull/31814 - Fused post-norm residual add into RMS NORM MUL and added set tensor n llama/31822 https://github.com/ggml-org/llama.cpp/pull/31822 , llama/31744 https://github.com/ggml-org/llama.cpp/pull/31744 - Added automatic libhtp loading and fixed 64-bit FastRPC handle truncation llama/31781 https://github.com/ggml-org/llama.cpp/pull/31781 , llama/30952 https://github.com/ggml-org/llama.cpp/pull/30952 - Fixed the host build with clang-cl on Windows ARM64 llama/32011 https://github.com/ggml-org/llama.cpp/pull/32011 CPU - Switched MUL MAT to F32 src1 and accumulate FlashAttention V in F32 llama/31854 https://github.com/ggml-org/llama.cpp/pull/31854 , llama/29810 https://github.com/ggml-org/llama.cpp/pull/29810 - Added SpacemiT GATED DELTA NET RVV and Q4 1 32x256 HP kernels and enabled IME2 MXFP4 repack llama/31897 https://github.com/ggml-org/llama.cpp/pull/31897 , llama/31929 https://github.com/ggml-org/llama.cpp/pull/31929 , llama/31776 https://github.com/ggml-org/llama.cpp/pull/31776 - Enabled the 4x4 simd gemm tile on ARM NEON and added bf16/q8 0 VXE to llamafile llama/32128 https://github.com/ggml-org/llama.cpp/pull/32128 , llama/31866 https://github.com/ggml-org/llama.cpp/pull/31866 CANN - Fail early when a buffer type is unsupported llama/31270 https://github.com/ggml-org/llama.cpp/pull/31270 ZDNN - Accept non-zdnn tensors, invalidate compute tensors after each op, and fix release backend lib detection llama/31824 https://github.com/ggml-org/llama.cpp/pull/31824 , llama/31905 https://github.com/ggml-org/llama.cpp/pull/31905 OpenVINO - Supported the YaRN truncation config llama/30802 https://github.com/ggml-org/llama.cpp/pull/30802 More info Changelog since v0.29.0 https://github.com/ggml-org/ggml/releases/tag/v0.29.0 417a024 https://github.com/ggml-org/ggml/commit/417a024b4fb6bc3116d41af7c067166a039bdf12 ggml : bump version to 1.0.0 1690 https://github.com/ggml-org/ggml/pull/1690 771b3d9 https://github.com/ggml-org/ggml/commit/771b3d9c45f26d046f1149a37ad01524341857e2 requirements : fix numpy for python 3.14 1691 https://github.com/ggml-org/ggml/pull/1691 f154959 https://github.com/ggml-org/ggml/commit/f154959e7f1c33a4237c67783fdd360b646eeb89 sync : llama.cpp ec04faf https://github.com/ggml-org/ggml/commit/ec04faf176635c4edd0d56a7f534e6d7a94f3a22 fix hexagon : make host build work with clang-cl on Windows ARM64 llama/32011 b45738e https://github.com/ggml-org/ggml/commit/b45738e946d6b3eb379d2505afc2b286357591dd examples : adapt to removal of ggml diag mask inf 1514439 https://github.com/ggml-org/ggml/commit/1514439034c6cf8cf5d5f98a82a6d156f3786443 sync : llama.cpp 5f29555 https://github.com/ggml-org/ggml/commit/5f2955545ae1f454c671834d3a07b1457fe2b65f ggml : prune llama/29801 076861c https://github.com/ggml-org/ggml/commit/076861cdfcc5b37ffbbcaa392513786bce9d5b70 hexagon: add Q5 0 and Q5 1 support llama/32094 63ec52d https://github.com/ggml-org/ggml/commit/63ec52d4f157f41d93aeb6ff792696f11827b187 Title: CUDA: few-row MMQ for BS 2-24 llama/31533 c6fe796 https://github.com/ggml-org/ggml/commit/c6fe79689e0c7682e182b87c1a0c3d1de55c8828 ggml : route GATED DELTA NET to the SpacemiT RVV kernel llama/31897 4ae6e72 https://github.com/ggml-org/ggml/commit/4ae6e7214d33a683056bac70668bb07d977fbace vulkan : Intel FA kernel for prefill. llama/29357 b22071c https://github.com/ggml-org/ggml/commit/b22071c095ac15a20b58e696e7a1a15b63882935 ggml-cpu: use the 4x4 simd gemm tile on ARM NEON llama/32128 ba8b013 https://github.com/ggml-org/ggml/commit/ba8b0130435ca0c327acefffc0427b2d4aa9df6a CUDA: Optimize prefil via fuse of w s scale in epilogue MMQ for nvfp4 checkpoints llama/26048 05f605a https://github.com/ggml-org/ggml/commit/05f605a25e91c30c5c85621d55f2cfb661ab7eec ggml-backend : prefetch only inputs computed by earlier splits llama/32100 b8a5b87 https://github.com/ggml-org/ggml/commit/b8a5b871f9445d6e0ad76d18ee592b5608d1ba74 ggml : dispatch SpacemiT Q4 1 32x256 through the HP kernel llama/31929 7a8cef5 https://github.com/ggml-org/ggml/commit/7a8cef580b1078e45df3864a88cb727a26b4d0c1 sycl: make oneDNN FA cache thread-safe and memoize runtime failures llama/31138 6dfea1a https://github.com/ggml-org/ggml/commit/6dfea1a1c5a30c3effc70bafb8692fac4f9f193d sycl : fix AOT for more than 4GB llama/31873 012ff44 https://github.com/ggml-org/ggml/commit/012ff44a3c532e6dc61a61be6e111df5a1ecd19e sycl : use cached device-id in set main device llama/32013 2f9d47b https://github.com/ggml-org/ggml/commit/2f9d47bb172e8dddb6a5010cf45579b71e608895 CUDA: add cuda debug buffer pool llama/31919 522d257 https://github.com/ggml-org/ggml/commit/522d2575e135aa0a39a887fdceebc9573dccec17 opencl: add bin kernel kernel gemm noshuffle q4 0 f32 32b trans ila a7 bin llama/30947 0f9dcb7 https://github.com/ggml-org/ggml/commit/0f9dcb72e3b1fc31f307885cd44a586a76c5cbb1 CUDA: skip empty MoE tiles in MMQ stream-k fixup llama/32067 6bed9a8 https://github.com/ggml-org/ggml/commit/6bed9a83c1cf9273d0ce4f5f1f2fbe1d323f4c28 opencl: add decomposed FA for Adreno llama/32046 a357e7b https://github.com/ggml-org/ggml/commit/a357e7bcddb7d7a65a58c138aaa4df6fd91c580e metal : prevent out-of-bounds reads in SOLVE TRI kernel llama/32103 c36ccd8 https://github.com/ggml-org/ggml/commit/c36ccd88f56b15cc000e3b34012db5784442b933 vulkan: support batched tensor copy API llama/31418 1bffc59 https://github.com/ggml-org/ggml/commit/1bffc59d478c8d0908d9e6e0d1a5887034a5e1fa backend : re-instantiate CUDA graphs when the cluster dims of a kernel change llama/31284 8153155 https://github.com/ggml-org/ggml/commit/8153155693491f4e17c825e3dec2ef04345e52e6 ROCm/HIP: fixed BLAS workspace instead of rocBLAS allocating one per call llama/31838 ad7d200 https://github.com/ggml-org/ggml/commit/ad7d200d601b25ceb6a003c7b80d6370a01d4cd2 cpu: accumulate flash attention V in F32 llama/29810 7f4a2db https://github.com/ggml-org/ggml/commit/7f4a2db327ccb3f517ac030a53106b3bcdd01eba webgpu: add PAD REFLECT 1D support llama/31877 0f51454 https://github.com/ggml-org/ggml/commit/0f514548be4eb0d0ef99a3f7ee9d9fcc992bb8a1 vulkan: allow dequantized KV optimization with CM2 llama/31526 eb4de49 https://github.com/ggml-org/ggml/commit/eb4de491b5d17b161e25fada5853770aba19d406 vulkan: allocate the device fault vendor binary buffer llama/32008 6b7f47a https://github.com/ggml-org/ggml/commit/6b7f47aae3964d006e334c44f317a7757681edb7 cuda : use MMVF for BF16 matrices of few rows at small batch sizes llama/31959 77d01f1 https://github.com/ggml-org/ggml/commit/77d01f1ba9df7ec52a672010a4361807c0ad7eb1 cuda : add sm70 FlashAttention config case for DKQ=256, DV=256, ncols=64 llama/27997 bd37ba0 https://github.com/ggml-org/ggml/commit/bd37ba0dfd4bbedc63d2178f844e9488a52d5b04 cuda : fattn mma head sizes 40 72 llama/30285 161ca15 https://github.com/ggml-org/ggml/commit/161ca156a3846b5d176289c5196554b84baafc94 CUDA: Add OCP FP8 support llama/28413 06faf80 https://github.com/ggml-org/ggml/commit/06faf806f1b4746b01e6678f39172797a6e4d8bc sycl : fixed sycl CLAMP to actually support strided view llama/31784 a986181 https://github.com/ggml-org/ggml/commit/a986181fe35e83cdfd7b2e8e7fa79a42cfe48996 cuda : remove unused ids and type src0 parameters llama/32018 55104b2 https://github.com/ggml-org/ggml/commit/55104b293feb7f1cc2091cc9aa119df06ce38573 llama : Add FP8 KV-cache support llama/28392 0753641 https://github.com/ggml-org/ggml/commit/0753641f42d02d3191ff1554bcdfb5b0cf89e521 ggml : add MUL MAT src1 precision tests and gating llama/31896 1cd16bd https://github.com/ggml-org/ggml/commit/1cd16bd4aabb5384f48d269cec751094266c3269 cuda : remove unused quantize cuda t typedef llama/31884 4a18cdd https://github.com/ggml-org/ggml/commit/4a18cdd8c807930c5d59aa3a079c034b845ebd1c cuda : halve VMM pool reservation size and retry on cuMemAddressReserve failure llama/30542 1c9ac9d https://github.com/ggml-org/ggml/commit/1c9ac9d88709fbc5287b09ebd582cf56ef6be50a CUDA: skip graph compat check when graph is being reused llama/32004 999c725 https://github.com/ggml-org/ggml/commit/999c7250ec311ab031e96e1cae85876aa064a0d6 ggml : fix FreeBSD Vulkan build by setting XOPEN SOURCE=700 llama/31701 47bc1d8 https://github.com/ggml-org/ggml/commit/47bc1d845c141a6fe63244d01140c7248424d126 CUDA: move graph and concurrent stream state out of common.cuh llama/32001 df70e12 https://github.com/ggml-org/ggml/commit/df70e121dfa94910f616519766d40c02fd9f5f06 cann: fail early when checking if buffer type is supported llama/31270 e447eb5 https://github.com/ggml-org/ggml/commit/e447eb53592717b491d3b6f8a4b1f270af2f85f9 sycl: Add device-side MoE scheduler llama/30156 9a1c582 https://github.com/ggml-org/ggml/commit/9a1c5822a7b8cd015ef83b663cd2f91c04f50ff7 vulkan: add tiled-transpose concat shader for Qwen models llama/30591 8b3c8cf https://github.com/ggml-org/ggml/commit/8b3c8cf97fdeb53dd151a7f4189e1937d7600b51 ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes llama/30415 f860f1d https://github.com/ggml-org/ggml/commit/f860f1ddb17a4a78235b78100cb6b7ac8e016e59 vulkan: Divide GROUP NORM workgroup to 2D to avoid maxComputeWorkGroupCount limit llama/31930 eb5571a https://github.com/ggml-org/ggml/commit/eb5571a3c9952efe261506f259e352ce70239f1c vulkan: Fix out of bounds reads/writes on MoltenVK llama/31890 5a9a264 https://github.com/ggml-org/ggml/commit/5a9a2643642264c51b301dba2b916fa7e68f6520 vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm llama/30829 5610f41 https://github.com/ggml-org/ggml/commit/5610f41bb6883a57198075a250354aa8a6893339 sycl: un-gate oneDNN fused SDPA for Alchemist at head dim=64 llama/31574 487cadf https://github.com/ggml-org/ggml/commit/487cadf89e0eeae16926ca3e00be2ad6667f454f hexagon: fix 64-bit FastRPC handle truncation on close llama/30952 be37c9c https://github.com/ggml-org/ggml/commit/be37c9cf04eba02a537e313579f5870357540689 ggml-zdnn: accept non-zdnn tensors, invalidate compute tensors after every op llama/31824 294d5fc https://github.com/ggml-org/ggml/commit/294d5fc5121ad278a4ed1e9a311f856308203e53 ggml : implement the backward pass for TRI llama/30712 e18c2c6 https://github.com/ggml-org/ggml/commit/e18c2c600a881d10549cf801c63c80a4ce98cba7 ggml-backend: don't upload inputs that are not read llama/31419 e84d36b https://github.com/ggml-org/ggml/commit/e84d36b54b61b91663ccf64dc1e7820e5f2e5839 metal : prevent out-of-bounds writes to the Metal FA output buffer llama/31662 0dd184f https://github.com/ggml-org/ggml/commit/0dd184f8b4e082e3c9fb2a37493aa69544dc316f metal : make ggml tri select instead of scaling by the comparison llama/30711 b6d482c https://github.com/ggml-org/ggml/commit/b6d482c58b3f24982547d667f1806640c62d126b opencl : include climits llama/31935 62de780 https://github.com/ggml-org/ggml/commit/62de7806fdcdda976bea593f8df0f84aaba54d77 RPC: Improve loading times llama/29413 a7276d0 https://github.com/ggml-org/ggml/commit/a7276d0ace8ea2b39695a50b6df4906d69647566 ggml : support YaRN truncation config on openvino, sycl, vulkan, webgpu, rpc llama/30802 da89459 https://github.com/ggml-org/ggml/commit/da894596c375d437a283f64ed2f408223335585f sycl: cache device-id llama/31126 fb57085 https://github.com/ggml-org/ggml/commit/fb570857a9be9613870878f155877aa7a1b7b3e4 tests: bound the test rope tolerance by the f32 angle resolution llama/31874 e43d847 https://github.com/ggml-org/ggml/commit/e43d8477bf7ef3dc9bddf98ed799b8b3628c841f opencl: add f32 MUL MAT ID llama/31783 d1c1e4b https://github.com/ggml-org/ggml/commit/d1c1e4b2c5d8f305f7d2e38bb5767391cf0cc902 ggml-zdnn: fix release binary not detecting backend lib llama/31905 7e7b6e9 https://github.com/ggml-org/ggml/commit/7e7b6e91954f1a574a1b78ec6302539e56d5f313 mtmd: refactoring: remove clip.h, consolidate code into mtmd llama/31865 5a0e536 https://github.com/ggml-org/ggml/commit/5a0e5368441ce74a30837831386e5bbafad9b596 cpu: enable SpacemiT IME2 MXFP4 dense repack llama/31776 7f4b25f https://github.com/ggml-org/ggml/commit/7f4b25f743ec2b069b3ffd2d3176982a0bf20504 llamafile: impl bf16 and q8 0 vxe llama/31866 f22fc6a https://github.com/ggml-org/ggml/commit/f22fc6a2c4bc87f10ae713e91dd9258701048b9d ggml : update CUDA/Metal radix Top-K to make tied selections deterministic llama/31084 47b8b4f https://github.com/ggml-org/ggml/commit/47b8b4fc33912bc26fcc6a8847e06b21d35487df ggml-cuda : add d state=64 fast path to SSM SCAN kernel llama/31818 b524960 https://github.com/ggml-org/ggml/commit/b524960690cab0cd7e3d57ac0946ed8519e4abe7 webgpu : add COUNT EQUAL support llama/31578 75a4f5a https://github.com/ggml-org/ggml/commit/75a4f5abaa727eb14409f381f72b317f8aeff537 ggml : switch to F32 src1 in the CPU backend llama/31854 6cddab2 https://github.com/ggml-org/ggml/commit/6cddab2b9c0a96ec0e97f710579fefdba9c5b234 llama, mtmd: follow the reference precision of ggml prec set acc calls llama/31851 7353275 https://github.com/ggml-org/ggml/commit/7353275dff29815336ddb91c7e050e7b78f43342 opencl: improve q8 0 and MoE small-n verification for spec llama/31831 8f1404e https://github.com/ggml-org/ggml/commit/8f1404ef548189ca05d43dc55e46055104eeabd3 hexagon: support for automatic loading of libhtp without legacy env vars llama/31781 91c102a https://github.com/ggml-org/ggml/commit/91c102a6874b8053ca914d306cee6064fb231186 sycl: exact rounding in CPY to Q4 1, Q5 1 and Q8 0 llama/31465 8cf32bc https://github.com/ggml-org/ggml/commit/8cf32bc64abfd1beb91c78283f00d6d534a4d919 sycl: refuse device buffers that would exceed VRAM llama/31285 d969b07 https://github.com/ggml-org/ggml/commit/d969b07e20e976d2b9e9a79a65d4c0a6812904fa hexagon: n hvx and n threads cleanup llama/31857 5746ebd https://github.com/ggml-org/ggml/commit/5746ebd6dd1324269afba5afaa3986147c729e5f llama moe cache: run missed experts on CPU, add tuning script llama/30966 a49f174 https://github.com/ggml-org/ggml/commit/a49f174db8e26c97f80faa818d91b47850b9c3bd sycl: add f16 support for SWIGLU OAI llama/30857 c6a0780 https://github.com/ggml-org/ggml/commit/c6a0780990ee695e12c1f64d7784b1fc76c30e0c metal : fix GROUP NORM for ne03 1 and non-contiguous inputs llama/31852 1e5c3e0 https://github.com/ggml-org/ggml/commit/1e5c3e03314ca8ed365053c36bd18a999f7e7207 webgpu : Implement group norm kernel llama/31490 37ecdca https://github.com/ggml-org/ggml/commit/37ecdca3c55e11367f61ac9265099e7d65b75451 vulkan: Add 'readonly' to decodeBuf declarations. llama/31833 c08538b https://github.com/ggml-org/ggml/commit/c08538b3a454b4c857084c6b75de30208eee8adf CUDA: fix shared memory bank conflicts in the lightning indexer llama/31803 07b8362 https://github.com/ggml-org/ggml/commit/07b8362f961c65b5c8e719c9622d12d318994e66 opencl: add arange llama/31829 f55a504 https://github.com/ggml-org/ggml/commit/f55a5045a70e11bd57ea80c0ee6d0d92d5537f90 meta : raise the compute view limit for buffers with few tensors llama/31447 2870133 https://github.com/ggml-org/ggml/commit/2870133190dbd494804455a5cf7a0d3ebd9efdaf hexagon: fuse post-norm residual add into RMS NORM MUL llama/31822 a135ec6 https://github.com/ggml-org/ggml/commit/a135ec677f7a3af22243096837eab82fb5e4b154 hexagon: conv 2d optimizations and support for conv 3d llama/31814 e3d3633 https://github.com/ggml-org/ggml/commit/e3d36332b33a302d679daa9d5da36f8e83b1ea65 tests : qualify isnan and signbit with std:: llama/31791 a7d3330 https://github.com/ggml-org/ggml/commit/a7d3330a46a9bba8c99fa6781c0e513355d19f01 tests : override hardware concurrency in CI llama/31797 ab7ca9c https://github.com/ggml-org/ggml/commit/ab7ca9c62d919a2787fa127598efda7001cd8f4f opencl: support repeat for f16, i32, i16, bf16 llama/31782 9290392 https://github.com/ggml-org/ggml/commit/9290392e451010eed8c6efc39172bcdf793d306f webgpu: add batched tensor API get tensor n and set tensor n llama/31473 f594fed https://github.com/ggml-org/ggml/commit/f594fedf92ca0b49240be326aed6308d920cbe18 vulkan: detect ReBAR support, avoid BAR allocations if unavailable llama/31658 38e4be2 https://github.com/ggml-org/ggml/commit/38e4be2bf02d8949101c299a38dc141de0e980c7 CUDA: fix Maxwell build llama/31787 8bd6b61 https://github.com/ggml-org/ggml/commit/8bd6b61b7a91b3f19eca5916e3e91baf38d6913e opencl: add q6 k multi column tiled GEMV for speculative verification and refactoring llama/31758 d3ce63d https://github.com/ggml-org/ggml/commit/d3ce63de46431e45d1d6085802c8294b067747c2 hexagon: Implement set tensor n and improve repacking/tiling robustness llama/31744 9b1880f https://github.com/ggml-org/ggml/commit/9b1880f93a7aaa5301945b80f7923f5674a5990a ggml-cuda: sanitize post-bias NaNs in MoE selection llama/31005 7e607b8 https://github.com/ggml-org/ggml/commit/7e607b8fee548b3f68ef4ba4af4eb588c412f570 vulkan: Implement test-fusion interface llama/30769 2188d13 https://github.com/ggml-org/ggml/commit/2188d132ce259b4ced3057639c98473903280ec2 CUDA: use highest compiled arch in get mmvq mmid max batch llama/31679 114ce5e https://github.com/ggml-org/ggml/commit/114ce5ed7e78d693403ae4e1b70f58e592ac5943 vulkan: RDNA3/4 Large-Tile Split-K Optimization llama/30280 e074347 https://github.com/ggml-org/ggml/commit/e074347a0c89f9eca517ff9aa47ad195243cc16e metal : fuse DeepSeek-V4/V4.1 and GLM MoE routing llama/31615 f0f5b42 https://github.com/ggml-org/ggml/commit/f0f5b42042d9ddaf0bda406441cb704e9807366b CUDA/HIP: matrix-core MFMA lightning indexer for CDNA2 gfx90a llama/29050 fa8c617 https://github.com/ggml-org/ggml/commit/fa8c6179b3663e16bcefc475e909bda73f901014 CUDA: linearize MoE data layout for MMQ/MMF llama/31047 1690d2e https://github.com/ggml-org/ggml/commit/1690d2e166a69438d26a5fb646d620f039396437 vulkan: FWHT kernels for block widths above 512 llama/29772 40278c4 https://github.com/ggml-org/ggml/commit/40278c41a43a1162a2c348a1b57ec0b970e80413 vulkan : check combined K+V f16 scratch size in FA dequant path llama/30487 2dccfdb https://github.com/ggml-org/ggml/commit/2dccfdb222d1ac791cc9f3461a078ce278019f1d sync : llama.cpp 25b845e https://github.com/ggml-org/ggml/commit/25b845e8d88870e85ba2176161f9da3d41f90771 ggml : support truncate=false for YaRN llama/30206 accf2a1 https://github.com/ggml-org/ggml/commit/accf2a1fbc78993310a61a1cd3829b17c66f5498 sync : llama.cpp b2ce926 https://github.com/ggml-org/ggml/commit/b2ce926cd4b46a23fb08d384037a9c9c11844984 metal : use mul mm for 32 <= K < 64 llama/30982 1a5f477 https://github.com/ggml-org/ggml/commit/1a5f4771609c9b6f7308a293611168085a06d726 webgpu : add MEAN support llama/31591 2ab6d6c https://github.com/ggml-org/ggml/commit/2ab6d6c44e5050ad29f227b8c60ed448f5391d02 vulkan: Re-enable large coopmat matmul configuration on AMD proprietary driver 26.10.1+ llama/31587 19df6cf https://github.com/ggml-org/ggml/commit/19df6cfea5755a9e3dce2923b832edeab9d56e4c tests: fix CI flakes of TOPK MOE caused by near ties llama/31426 a0df99d https://github.com/ggml-org/ggml/commit/a0df99d731a720caf097fc93765b097d29c34014 metal: keep fusable node groups inside one command buffer llama/31688 3a996db https://github.com/ggml-org/ggml/commit/3a996dba9b1a025b75cffb3c7006ea3fb193ffd4 opencl: enable f32 and q5 k multi column GEMV by default for Adreno X2 llama/31694 d75184e https://github.com/ggml-org/ggml/commit/d75184ebf244d53f9074cd4f5e7320239ce0e466 opencl: support non-contig sigmoid llama/31687 c634501 https://github.com/ggml-org/ggml/commit/c6345010f9010fa01d790507abfd8f7a8fbe6b18 opencl: add DSV4 HC PRE and DSV4 HC POST llama/31623 0816137 https://github.com/ggml-org/ggml/commit/08161372fcfa4aa545a0f346d7d15af00f433f30 Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ llama/31296 4ac0dae https://github.com/ggml-org/ggml/commit/4ac0dae8acb8c8a66f10051394af051d48db939f opencl: improve decode perf with FA llama/31633 4f681f2 https://github.com/ggml-org/ggml/commit/4f681f28cbafc27f7a1e916d0f50092963de1437 webgpu: return maxBufferSize from get max size and avoid whole-buffer bindings in memset/clear llama/31348 cd6d2b7 https://github.com/ggml-org/ggml/commit/cd6d2b75f5d3e4b2f3fe8e8fbc6c8b84ac80e74a docs : sync ggml type enum in gguf.md 1685 https://github.com/ggml-org/ggml/pull/1685