llama.cpp patch speeds up one RTX 5090 matrix multiply 4.5x

Llama.cpp merged new Vulkan cooperative-matrix tiles on November 14th, 2026, cutting one Q4_K matrix multiply at M=4096, K=14336, N=16 on an NVIDIA GeForce RTX 5090 from 325.49 microseconds to 72.36 microseconds in the KHR cooperative-matrix path, a 4.50x kernel speedup reported in the merged pull request. The patch, attributed to Jeff Bolz, uses narrow tiles to target that specific shape in the inference project created by Georgi Gerganov.

llama.cpp patch speeds up one RTX 5090 matrix multiply 4.5x Jeff Bolz's narrow tiles cut the KHR cooperative-matrix case at M=4096, K=14336, N=16 from 325.49 to 72.36 microseconds on an RTX 5090. By RuntimeWire Staff https://runtimewire.com/author/runtimewire-staff · Published llama.cpp, the inference project created by Georgi Gerganov, merged new Vulkan cooperative-matrix tiles on November 14th, 2026. The sharpest result was for one matrix shape: Q4 K, M=4096, K=14336, N=16, on an NVIDIA GeForce RTX 5090. In the KHR cooperative-matrix path, that multiply fell from 325.49 microseconds to 72.36 microseconds, a 4.50x kernel speedup reported in the merged pull request.…