ktransformers

A hobbyist quantizer known as Aesedai produced a GLM-5.2 GGUF quant using a 512GB RAM workstation with 2x56-core Emerald Rapids Xeons and a single RTX 6000, achieving about 15 tokens per second at 16k…

TurboFieldfare, a Swift-and-Metal runtime, runs Google's Gemma 4 26B-A4B model on Apple Silicon Macs with as little as 2 GB of RAM by streaming expert weights from SSD, achieving 5.1–6.3 tokens per se…

A developer achieved ~28 tok/s single-stream decode of DeepSeek-V4-Flash (284B MoE) on a single RTX 5090 with dual Xeon Gold 6138 CPUs using a CPU-offloaded expert recipe. The setup uses ktransformers…