Ggml v1.0.0

wpnews.pro

Overview #

ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quantized matmuls and MoE fusion on CUDA/Metal, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, new WebGPU kernels, expanded Hexagon op coverage, and scheduler improvements that avoid up unused inputs.

API changes

Pruned deprecated API and removed the legacy DIAG_MASK_INF /DIAG_MASK_ZERO ops, bumping the RPC protocol to reject incompatible peers (llama/29801 ,llama/31946 )
Added ggml_rope_set_truncate /ggml_rope_get_truncate to control YaRN correction-bound rounding (llama/30206 ,llama/30802 )
Implemented the backward pass for TRI (llama/30712 )
Added MUL_MAT src1 precision tests and gating forggml_prec_set_acc (llama/31896 ,llama/31851 )
Bumped the RPC protocol to v9 with faster and graceful failure on unreachable endpoints or failed handshakes (llama/29413 ,llama/30415 )

Core changes

Scheduler skips up inputs that are not read and prefetches only inputs produced by earlier splits (llama/31419 ,llama/32100 )

- MoE cache can compute a share of the cache misses on the CPU and copy the results back asynchronously ([llama/30966](https://github.com/ggml-org/llama.cpp/pull/30966) )
- Meta backend raises the compute view limit for buffers with few tensors ([llama/31447](https://github.com/ggml-org/llama.cpp/pull/31447) )

Backend changes

CUDA

Added few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls (llama/31533 ,llama/31959 )
Added OCP FP8 support and fused NVFP4 gate/up matmuls with GLU in MMQ (llama/28413 ,llama/31296 )
Extended FlashAttention with an sm70 head-size-256 config and MMA head sizes 40/72 (llama/27997 ,llama/30285 )
Improved graph handling: skip compat checks on reuse, re-instantiate on cluster-dim changes, move graph state (llama/32004 ,llama/31284 ,llama/32001 )
MoE: skip empty MMQ stream-k tiles, linearize the MoE layout, and sanitize post-bias NaNs (llama/32067 ,llama/31047 ,llama/31005 )
Added a debug buffer pool and halved the VMM pool reservation with reserve retries (llama/31919 ,llama/30542 )
HIP: use a fixed BLAS workspace and add an MFMA lightning indexer for CDNA2 (llama/31838 ,llama/29050 )
Added a d_state=64 SSM_SCAN fast path and made radix Top-K deterministic for ties (llama/31818 ,llama/31084 )

Metal

Fused MoE routing and kept fusable node groups inside a single command buffer (llama/31615 ,llama/31688 )
Made radix Top-K deterministic for ties, added mul_mm for 32<=K<64, and switched TRI to select (llama/31084 ,llama/30982 ,llama/30711 )
Fixed GROUP_NORM for ne03>1 and out-of-bounds accesses in the FA output and SOLVE_TRI kernels (llama/31852 ,llama/31662 ,llama/32103 )

Vulkan

Added an Intel FlashAttention prefill kernel (llama/29357 )
Added batched tensor copies, a test-fusion interface, and ReBAR detection (llama/31418 ,llama/30769 ,llama/31658 )
Enabled dequantized KV with CM2, YaRN truncation, RDNA3/4 large-tile split-K, and large coopmat matmul on newer AMD drivers (llama/31526 ,llama/30802 ,llama/30280 ,llama/31587 )
Added a tiled-transpose concat shader and FWHT kernels for block widths above 512 (llama/30591 ,llama/29772 )
Fixed MoltenVK out-of-bounds accesses, GROUP_NORM workgroup limits, and FA scratch sizing (llama/31890 ,llama/31930 ,llama/30487 )
Capped scalar FlashAttention at 8 rows for large heads on Qualcomm (llama/30829 )

SYCL

Added a device-side MoE scheduler (llama/30156 )
Made the oneDNN FA cache thread-safe and un-gated fused SDPA for Alchemist at head_dim=64 (llama/31138 ,llama/31574 )
Cached the device id and reject buffers that would exceed VRAM (llama/31126 ,llama/32013 ,llama/31285 )
Added exact rounding for CPY to Q4_1/Q5_1/Q8_0, strided CLAMP, and f16 SWIGLU_OAI (llama/31465 ,llama/31784 ,llama/30857 )
Fixed AOT for allocations above 4GB (llama/31873 )

OpenCL

Added decomposed Adreno FlashAttention and improved decode performance (llama/32046 ,llama/31633 )
Added q6_k tiled, q8_0, and f32/q5_k multi-column GEMV kernels and f32 MUL_MAT_ID (llama/31758 ,llama/30947 ,llama/31783 )
Added arange, non-contiguous sigmoid, repeat for f16/i32/i16/bf16, and DSV4 HC pre/post kernels (llama/31829 ,llama/31687 ,llama/31782 ,llama/31623 )
Enabled multi-column GEMV by default for Adreno X2 (llama/31694 )

WebGPU

Added batched tensor API support and returned maxBufferSize from get_max_size (llama/31473 ,llama/31348 )
Added GROUP_NORM, MEAN, COUNT_EQUAL, and PAD_REFLECT_1D kernels (llama/31490 ,llama/31591 ,llama/31578 ,llama/31877 )
Supported the YaRN truncation config (llama/30802 )

Hexagon

Added Q5_0/Q5_1 support and optimized conv_2d while adding conv_3d (llama/32094 ,llama/31814 )
Fused post-norm residual add into RMS_NORM_MUL and added set_tensor_n (llama/31822 ,llama/31744 )

- Added automatic libhtp  and fixed 64-bit FastRPC handle truncation ([llama/31781](https://github.com/ggml-org/llama.cpp/pull/31781) ,[llama/30952](https://github.com/ggml-org/llama.cpp/pull/30952) )
- Fixed the host build with clang-cl on Windows ARM64 ([llama/32011](https://github.com/ggml-org/llama.cpp/pull/32011) )

CPU

Switched MUL_MAT to F32 src1 and accumulate FlashAttention V in F32 (llama/31854 ,llama/29810 )
Added SpacemiT GATED_DELTA_NET RVV and Q4_1 32x256 HP kernels and enabled IME2 MXFP4 repack (llama/31897 ,llama/31929 ,llama/31776 )
Enabled the 4x4 simd_gemm tile on ARM NEON and added bf16/q8_0 VXE to llamafile (llama/32128 ,llama/31866 )

CANN

Fail early when a buffer type is unsupported (llama/31270 )

ZDNN

Accept non-zdnn tensors, invalidate compute tensors after each op, and fix release backend lib detection (llama/31824 ,llama/31905 )

OpenVINO

Supported the YaRN truncation config (llama/30802 )

More info #

## Changelog since [v0.29.0](https://github.com/ggml-org/ggml/releases/tag/v0.29.0)

[`417a024`](https://github.com/ggml-org/ggml/commit/417a024b4fb6bc3116d41af7c067166a039bdf12) ggml : bump version to 1.0.0 ([#1690](https://github.com/ggml-org/ggml/pull/1690))

[`771b3d9`](https://github.com/ggml-org/ggml/commit/771b3d9c45f26d046f1149a37ad01524341857e2) requirements : fix numpy for python 3.14 ([#1691](https://github.com/ggml-org/ggml/pull/1691))

[`f154959`](https://github.com/ggml-org/ggml/commit/f154959e7f1c33a4237c67783fdd360b646eeb89) sync : llama.cpp

[`ec04faf`](https://github.com/ggml-org/ggml/commit/ec04faf176635c4edd0d56a7f534e6d7a94f3a22) fix(hexagon): make host build work with clang-cl on Windows ARM64 (llama/32011)

b45738e examples : adapt to removal of ggml_diag_mask_inf

[`1514439`](https://github.com/ggml-org/ggml/commit/1514439034c6cf8cf5d5f98a82a6d156f3786443) sync : llama.cpp

[`5f29555`](https://github.com/ggml-org/ggml/commit/5f2955545ae1f454c671834d3a07b1457fe2b65f) ggml : prune (llama/29801)

[`076861c`](https://github.com/ggml-org/ggml/commit/076861cdfcc5b37ffbbcaa392513786bce9d5b70) hexagon: add Q5_0 and Q5_1 support (llama/32094)

[`63ec52d`](https://github.com/ggml-org/ggml/commit/63ec52d4f157f41d93aeb6ff792696f11827b187) Title: CUDA: few-row MMQ for BS 2-24 (llama/31533)

[`c6fe796`](https://github.com/ggml-org/ggml/commit/c6fe79689e0c7682e182b87c1a0c3d1de55c8828) ggml : route GATED_DELTA_NET to the SpacemiT RVV kernel (llama/31897)

[`4ae6e72`](https://github.com/ggml-org/ggml/commit/4ae6e7214d33a683056bac70668bb07d977fbace) vulkan : Intel FA kernel for prefill. (llama/29357)

[`b22071c`](https://github.com/ggml-org/ggml/commit/b22071c095ac15a20b58e696e7a1a15b63882935) ggml-cpu: use the 4x4 simd_gemm tile on ARM NEON (llama/32128)

ba8b013 CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (llama/26048)

[`05f605a`](https://github.com/ggml-org/ggml/commit/05f605a25e91c30c5c85621d55f2cfb661ab7eec) ggml-backend : prefetch only inputs computed by earlier splits (llama/32100)

[`b8a5b87`](https://github.com/ggml-org/ggml/commit/b8a5b871f9445d6e0ad76d18ee592b5608d1ba74) ggml : dispatch SpacemiT Q4_1 32x256 through the HP kernel (llama/31929)

[`7a8cef5`](https://github.com/ggml-org/ggml/commit/7a8cef580b1078e45df3864a88cb727a26b4d0c1) sycl: make oneDNN FA cache thread-safe and memoize runtime failures (llama/31138)

[`6dfea1a`](https://github.com/ggml-org/ggml/commit/6dfea1a1c5a30c3effc70bafb8692fac4f9f193d) sycl : fix AOT for more than 4GB (llama/31873)

[`012ff44`](https://github.com/ggml-org/ggml/commit/012ff44a3c532e6dc61a61be6e111df5a1ecd19e) sycl : use cached device-id in set_main_device (llama/32013)

[`2f9d47b`](https://github.com/ggml-org/ggml/commit/2f9d47bb172e8dddb6a5010cf45579b71e608895) CUDA: add cuda debug buffer pool (llama/31919)

522d257 opencl: add bin kernel kernel_gemm_noshuffle_q4_0_f32_32b_trans_ila_a7_bin (llama/30947)

[`0f9dcb7`](https://github.com/ggml-org/ggml/commit/0f9dcb72e3b1fc31f307885cd44a586a76c5cbb1) CUDA: skip empty MoE tiles in MMQ stream-k fixup (llama/32067)

[`6bed9a8`](https://github.com/ggml-org/ggml/commit/6bed9a83c1cf9273d0ce4f5f1f2fbe1d323f4c28) opencl: add decomposed FA for Adreno (llama/32046)

[`a357e7b`](https://github.com/ggml-org/ggml/commit/a357e7bcddb7d7a65a58c138aaa4df6fd91c580e) metal : prevent out-of-bounds reads in SOLVE_TRI kernel (llama/32103)

[`c36ccd8`](https://github.com/ggml-org/ggml/commit/c36ccd88f56b15cc000e3b34012db5784442b933) vulkan: support batched tensor copy API (llama/31418)

[`1bffc59`](https://github.com/ggml-org/ggml/commit/1bffc59d478c8d0908d9e6e0d1a5887034a5e1fa) backend : re-instantiate CUDA graphs when the cluster dims of a kernel change (llama/31284)

8153155 ROCm/HIP: fixed BLAS workspace instead of rocBLAS allocating one per call (llama/31838)

[`ad7d200`](https://github.com/ggml-org/ggml/commit/ad7d200d601b25ceb6a003c7b80d6370a01d4cd2) cpu: accumulate flash attention V in F32 (llama/29810)

[`7f4a2db`](https://github.com/ggml-org/ggml/commit/7f4a2db327ccb3f517ac030a53106b3bcdd01eba) webgpu: add PAD_REFLECT_1D support (llama/31877)

[`0f51454`](https://github.com/ggml-org/ggml/commit/0f514548be4eb0d0ef99a3f7ee9d9fcc992bb8a1) vulkan: allow dequantized KV optimization with CM2 (llama/31526)

[`eb4de49`](https://github.com/ggml-org/ggml/commit/eb4de491b5d17b161e25fada5853770aba19d406) vulkan: allocate the device fault vendor binary buffer (llama/32008)

[`6b7f47a`](https://github.com/ggml-org/ggml/commit/6b7f47aae3964d006e334c44f317a7757681edb7) cuda : use MMVF for BF16 matrices of few rows at small batch sizes (llama/31959)

[`77d01f1`](https://github.com/ggml-org/ggml/commit/77d01f1ba9df7ec52a672010a4361807c0ad7eb1) cuda : add sm70 FlashAttention config case for DKQ=256, DV=256, ncols=64 (llama/27997)

[`bd37ba0`](https://github.com/ggml-org/ggml/commit/bd37ba0dfd4bbedc63d2178f844e9488a52d5b04) cuda : fattn mma head sizes 40 72 (llama/30285)

[`161ca15`](https://github.com/ggml-org/ggml/commit/161ca156a3846b5d176289c5196554b84baafc94) CUDA: Add OCP FP8 support (llama/28413)

[`06faf80`](https://github.com/ggml-org/ggml/commit/06faf806f1b4746b01e6678f39172797a6e4d8bc) sycl : fixed sycl CLAMP to actually support strided view (llama/31784)

[`a986181`](https://github.com/ggml-org/ggml/commit/a986181fe35e83cdfd7b2e8e7fa79a42cfe48996) cuda : remove unused ids and type_src0 parameters (llama/32018)

[`55104b2`](https://github.com/ggml-org/ggml/commit/55104b293feb7f1cc2091cc9aa119df06ce38573) llama : Add FP8 KV-cache support (llama/28392)

[`0753641`](https://github.com/ggml-org/ggml/commit/0753641f42d02d3191ff1554bcdfb5b0cf89e521) ggml : add MUL_MAT src1 precision tests and gating (llama/31896)

[`1cd16bd`](https://github.com/ggml-org/ggml/commit/1cd16bd4aabb5384f48d269cec751094266c3269) cuda : remove unused quantize_cuda_t typedef (llama/31884)

4a18cdd cuda : halve VMM pool reservation size and retry on cuMemAddressReserve failure (llama/30542)

[`1c9ac9d`](https://github.com/ggml-org/ggml/commit/1c9ac9d88709fbc5287b09ebd582cf56ef6be50a) CUDA: skip graph compat check when graph is being reused (llama/32004)

[`999c725`](https://github.com/ggml-org/ggml/commit/999c7250ec311ab031e96e1cae85876aa064a0d6) ggml : fix FreeBSD Vulkan build by setting _XOPEN_SOURCE=700 (llama/31701)

[`47bc1d8`](https://github.com/ggml-org/ggml/commit/47bc1d845c141a6fe63244d01140c7248424d126) CUDA: move graph and concurrent stream state out of common.cuh (llama/32001)

[`df70e12`](https://github.com/ggml-org/ggml/commit/df70e121dfa94910f616519766d40c02fd9f5f06) cann: fail early when checking if buffer type is supported (llama/31270)

[`e447eb5`](https://github.com/ggml-org/ggml/commit/e447eb53592717b491d3b6f8a4b1f270af2f85f9) sycl: Add device-side MoE scheduler (llama/30156)

[`9a1c582`](https://github.com/ggml-org/ggml/commit/9a1c5822a7b8cd015ef83b663cd2f91c04f50ff7) vulkan: add tiled-transpose concat shader for Qwen models (llama/30591)

[`8b3c8cf`](https://github.com/ggml-org/ggml/commit/8b3c8cf97fdeb53dd151a7f4189e1937d7600b51) ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes (llama/30415)

f860f1d vulkan: Divide GROUP_NORM workgroup to 2D to avoid maxComputeWorkGroupCount limit (llama/31930)

eb5571a vulkan: Fix out of bounds reads/writes on MoltenVK (llama/31890) 5a9a264 vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm (llama/30829)

[`5610f41`](https://github.com/ggml-org/ggml/commit/5610f41bb6883a57198075a250354aa8a6893339) sycl: un-gate oneDNN fused SDPA for Alchemist at head_dim=64 (llama/31574)

[`487cadf`](https://github.com/ggml-org/ggml/commit/487cadf89e0eeae16926ca3e00be2ad6667f454f) hexagon: fix 64-bit FastRPC handle truncation on close (llama/30952)

[`be37c9c`](https://github.com/ggml-org/ggml/commit/be37c9cf04eba02a537e313579f5870357540689) ggml-zdnn: accept non-zdnn tensors, invalidate compute tensors after every op (llama/31824)

[`294d5fc`](https://github.com/ggml-org/ggml/commit/294d5fc5121ad278a4ed1e9a311f856308203e53) ggml : implement the backward pass for TRI (llama/30712)

[`e18c2c6`](https://github.com/ggml-org/ggml/commit/e18c2c600a881d10549cf801c63c80a4ce98cba7) ggml-backend: don't upload inputs that are not read (llama/31419)

[`e84d36b`](https://github.com/ggml-org/ggml/commit/e84d36b54b61b91663ccf64dc1e7820e5f2e5839) metal : prevent out-of-bounds writes to the Metal FA output buffer (llama/31662)

[`0dd184f`](https://github.com/ggml-org/ggml/commit/0dd184f8b4e082e3c9fb2a37493aa69544dc316f) metal : make ggml_tri select instead of scaling by the comparison (llama/30711)

[`b6d482c`](https://github.com/ggml-org/ggml/commit/b6d482c58b3f24982547d667f1806640c62d126b) opencl : include climits (llama/31935)

[`62de780`](https://github.com/ggml-org/ggml/commit/62de7806fdcdda976bea593f8df0f84aaba54d77) RPC: Improve  times (llama/29413)

a7276d0 ggml : support YaRN truncation config on openvino, sycl, vulkan, webgpu, rpc (llama/30802)

[`da89459`](https://github.com/ggml-org/ggml/commit/da894596c375d437a283f64ed2f408223335585f) sycl: cache device-id (llama/31126)

[`fb57085`](https://github.com/ggml-org/ggml/commit/fb570857a9be9613870878f155877aa7a1b7b3e4) tests: bound the test_rope tolerance by the f32 angle resolution (llama/31874)

[`e43d847`](https://github.com/ggml-org/ggml/commit/e43d8477bf7ef3dc9bddf98ed799b8b3628c841f) opencl: add f32 MUL_MAT_ID (llama/31783)

[`d1c1e4b`](https://github.com/ggml-org/ggml/commit/d1c1e4b2c5d8f305f7d2e38bb5767391cf0cc902) ggml-zdnn: fix release binary not detecting backend lib (llama/31905)

[`7e7b6e9`](https://github.com/ggml-org/ggml/commit/7e7b6e91954f1a574a1b78ec6302539e56d5f313) mtmd: refactoring: remove clip.h, consolidate code into mtmd (llama/31865)

[`5a0e536`](https://github.com/ggml-org/ggml/commit/5a0e5368441ce74a30837831386e5bbafad9b596) cpu: enable SpacemiT IME2 MXFP4 dense repack (llama/31776)

[`7f4b25f`](https://github.com/ggml-org/ggml/commit/7f4b25f743ec2b069b3ffd2d3176982a0bf20504) llamafile: impl bf16 and q8_0 vxe (llama/31866)

[`f22fc6a`](https://github.com/ggml-org/ggml/commit/f22fc6a2c4bc87f10ae713e91dd9258701048b9d) ggml : update CUDA/Metal radix Top-K to make tied selections deterministic (llama/31084)

[`47b8b4f`](https://github.com/ggml-org/ggml/commit/47b8b4fc33912bc26fcc6a8847e06b21d35487df) ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (llama/31818)

[`b524960`](https://github.com/ggml-org/ggml/commit/b524960690cab0cd7e3d57ac0946ed8519e4abe7) webgpu : add COUNT_EQUAL support (llama/31578)

[`75a4f5a`](https://github.com/ggml-org/ggml/commit/75a4f5abaa727eb14409f381f72b317f8aeff537) ggml : switch to F32 src1 in the CPU backend (llama/31854)

[`6cddab2`](https://github.com/ggml-org/ggml/commit/6cddab2b9c0a96ec0e97f710579fefdba9c5b234) llama, mtmd: follow the reference precision of ggml_prec_set_acc calls (llama/31851)

[`7353275`](https://github.com/ggml-org/ggml/commit/7353275dff29815336ddb91c7e050e7b78f43342) opencl: improve q8_0 and MoE small-n verification for spec (llama/31831)

[`8f1404e`](https://github.com/ggml-org/ggml/commit/8f1404ef548189ca05d43dc55e46055104eeabd3) hexagon: support for automatic  of libhtp without legacy env vars (llama/31781)

[`91c102a`](https://github.com/ggml-org/ggml/commit/91c102a6874b8053ca914d306cee6064fb231186) sycl: exact rounding in CPY to Q4_1, Q5_1 and Q8_0 (llama/31465)

[`8cf32bc`](https://github.com/ggml-org/ggml/commit/8cf32bc64abfd1beb91c78283f00d6d534a4d919) sycl: refuse device buffers that would exceed VRAM (llama/31285)

[`d969b07`](https://github.com/ggml-org/ggml/commit/d969b07e20e976d2b9e9a79a65d4c0a6812904fa) hexagon: n_hvx and n_threads cleanup (llama/31857)

[`5746ebd`](https://github.com/ggml-org/ggml/commit/5746ebd6dd1324269afba5afaa3986147c729e5f) llama_moe_cache: run missed experts on CPU, add tuning script (llama/30966)

[`a49f174`](https://github.com/ggml-org/ggml/commit/a49f174db8e26c97f80faa818d91b47850b9c3bd) sycl: add f16 support for SWIGLU_OAI (llama/30857)

[`c6a0780`](https://github.com/ggml-org/ggml/commit/c6a0780990ee695e12c1f64d7784b1fc76c30e0c) metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs (llama/31852)

[`1e5c3e0`](https://github.com/ggml-org/ggml/commit/1e5c3e03314ca8ed365053c36bd18a999f7e7207) webgpu : Implement group norm kernel (llama/31490)

[`37ecdca`](https://github.com/ggml-org/ggml/commit/37ecdca3c55e11367f61ac9265099e7d65b75451) vulkan: Add 'readonly' to decodeBuf declarations. (llama/31833)

[`c08538b`](https://github.com/ggml-org/ggml/commit/c08538b3a454b4c857084c6b75de30208eee8adf) CUDA: fix shared memory bank conflicts in the lightning indexer (llama/31803)

[`07b8362`](https://github.com/ggml-org/ggml/commit/07b8362f961c65b5c8e719c9622d12d318994e66) opencl: add arange (llama/31829)

[`f55a504`](https://github.com/ggml-org/ggml/commit/f55a5045a70e11bd57ea80c0ee6d0d92d5537f90) meta : raise the compute view limit for buffers with few tensors (llama/31447)

[`2870133`](https://github.com/ggml-org/ggml/commit/2870133190dbd494804455a5cf7a0d3ebd9efdaf) hexagon: fuse post-norm residual add into RMS_NORM_MUL (llama/31822)

[`a135ec6`](https://github.com/ggml-org/ggml/commit/a135ec677f7a3af22243096837eab82fb5e4b154) hexagon: conv_2d optimizations and support for conv_3d (llama/31814)

[`e3d3633`](https://github.com/ggml-org/ggml/commit/e3d36332b33a302d679daa9d5da36f8e83b1ea65) tests : qualify isnan and signbit with std:: (llama/31791)

[`a7d3330`](https://github.com/ggml-org/ggml/commit/a7d3330a46a9bba8c99fa6781c0e513355d19f01) tests : override hardware_concurrency in CI (llama/31797)

[`ab7ca9c`](https://github.com/ggml-org/ggml/commit/ab7ca9c62d919a2787fa127598efda7001cd8f4f) opencl: support repeat for f16, i32, i16, bf16 (llama/31782)

[`9290392`](https://github.com/ggml-org/ggml/commit/9290392e451010eed8c6efc39172bcdf793d306f) webgpu: add batched tensor API (`get_tensor_n` and `set_tensor_n`) (llama/31473)

[`f594fed`](https://github.com/ggml-org/ggml/commit/f594fedf92ca0b49240be326aed6308d920cbe18) vulkan: detect ReBAR support, avoid BAR allocations if unavailable (llama/31658)

[`38e4be2`](https://github.com/ggml-org/ggml/commit/38e4be2bf02d8949101c299a38dc141de0e980c7) CUDA: fix Maxwell build (llama/31787)

8bd6b61 opencl: add q6_k multi column tiled GEMV for speculative verification and refactoring (llama/31758)

[`d3ce63d`](https://github.com/ggml-org/ggml/commit/d3ce63de46431e45d1d6085802c8294b067747c2) hexagon: Implement set_tensor_n and improve repacking/tiling robustness (llama/31744)

[`9b1880f`](https://github.com/ggml-org/ggml/commit/9b1880f93a7aaa5301945b80f7923f5674a5990a) ggml-cuda: sanitize post-bias NaNs in MoE selection (llama/31005)

[`7e607b8`](https://github.com/ggml-org/ggml/commit/7e607b8fee548b3f68ef4ba4af4eb588c412f570) vulkan: Implement test-fusion interface (llama/30769)

[`2188d13`](https://github.com/ggml-org/ggml/commit/2188d132ce259b4ced3057639c98473903280ec2) CUDA: use highest compiled arch in get_mmvq_mmid_max_batch (llama/31679)

[`114ce5e`](https://github.com/ggml-org/ggml/commit/114ce5ed7e78d693403ae4e1b70f58e592ac5943) vulkan: RDNA3/4 Large-Tile Split-K Optimization (llama/30280)

[`e074347`](https://github.com/ggml-org/ggml/commit/e074347a0c89f9eca517ff9aa47ad195243cc16e) metal : fuse DeepSeek-V4/V4.1 and GLM MoE routing (llama/31615)

[`f0f5b42`](https://github.com/ggml-org/ggml/commit/f0f5b42042d9ddaf0bda406441cb704e9807366b) CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (llama/29050)

[`fa8c617`](https://github.com/ggml-org/ggml/commit/fa8c6179b3663e16bcefc475e909bda73f901014) CUDA: linearize MoE data layout for MMQ/MMF (llama/31047)

[`1690d2e`](https://github.com/ggml-org/ggml/commit/1690d2e166a69438d26a5fb646d620f039396437) vulkan: FWHT kernels for block widths above 512 (llama/29772)

[`40278c4`](https://github.com/ggml-org/ggml/commit/40278c41a43a1162a2c348a1b57ec0b970e80413) vulkan : check combined K+V f16 scratch size in FA dequant path (llama/30487)

[`2dccfdb`](https://github.com/ggml-org/ggml/commit/2dccfdb222d1ac791cc9f3461a078ce278019f1d) sync : llama.cpp

[`25b845e`](https://github.com/ggml-org/ggml/commit/25b845e8d88870e85ba2176161f9da3d41f90771) ggml : support `truncate=false` for YaRN (llama/30206)

[`accf2a1`](https://github.com/ggml-org/ggml/commit/accf2a1fbc78993310a61a1cd3829b17c66f5498) sync : llama.cpp

[`b2ce926`](https://github.com/ggml-org/ggml/commit/b2ce926cd4b46a23fb08d384037a9c9c11844984) metal : use mul_mm for 32 <= K < 64 (llama/30982)

[`1a5f477`](https://github.com/ggml-org/ggml/commit/1a5f4771609c9b6f7308a293611168085a06d726) webgpu : add MEAN support (llama/31591)

[`2ab6d6c`](https://github.com/ggml-org/ggml/commit/2ab6d6c44e5050ad29f227b8c60ed448f5391d02) vulkan: Re-enable large coopmat matmul configuration on AMD proprietary driver 26.10.1+ (llama/31587)

[`19df6cf`](https://github.com/ggml-org/ggml/commit/19df6cfea5755a9e3dce2923b832edeab9d56e4c) tests: fix CI flakes of TOPK_MOE caused by near ties (llama/31426)

[`a0df99d`](https://github.com/ggml-org/ggml/commit/a0df99d731a720caf097fc93765b097d29c34014) metal: keep fusable node groups inside one command buffer (llama/31688)

[`3a996db`](https://github.com/ggml-org/ggml/commit/3a996dba9b1a025b75cffb3c7006ea3fb193ffd4) opencl: enable f32 and q5_k multi column GEMV by default for Adreno X2 (llama/31694)

[`d75184e`](https://github.com/ggml-org/ggml/commit/d75184ebf244d53f9074cd4f5e7320239ce0e466) opencl: support non-contig sigmoid (llama/31687)

[`c634501`](https://github.com/ggml-org/ggml/commit/c6345010f9010fa01d790507abfd8f7a8fbe6b18) opencl: add DSV4_HC_PRE and DSV4_HC_POST (llama/31623)

[`0816137`](https://github.com/ggml-org/ggml/commit/08161372fcfa4aa545a0f346d7d15af00f433f30) Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ (llama/31296)

[`4ac0dae`](https://github.com/ggml-org/ggml/commit/4ac0dae8acb8c8a66f10051394af051d48db939f) opencl: improve decode perf with FA (llama/31633)

4f681f2 webgpu: return maxBufferSize from get_max_size and avoid whole-buffer bindings in memset/clear (llama/31348)

cd6d2b7 docs : sync ggml_type enum in gguf.md (#1685)

source & further reading

github.com — original article Open-source ChatGPT Enterprise alternative, built on the Codex runtime Show HN: Memro – Persistent Memory for Claude Code, Cursor and Codex Doc-foundry – AI writes Security detection rules, then proves they work