KTransformers 0.7, the heterogeneous LLM inference and fine-tuning framework, is now available with full AVX-512 support for LoRA fine-tuning that no longer requires Advanced Matrix Extensions (AMX), …
KTransformers, a Tsinghua-born framework now with 19k GitHub stars, has restructured from a standalone inference system into CPU kernels and placement strategies embedded within SGLang and LLaMA-Facto…
KTransformers, a flexible LLM inference framework, enables deployment of 100B+ parameter models locally on a single RTX 5090 (32GB VRAM) using CPU/GPU heterogeneous computing without quantization. The…
The MADSys Lab at Tsinghua University’s KTransformers project enables frontier-class MoE models like DeepSeek-R1 671B to run on commodity hardware with a CPU-GPU hybrid inference stack, achieving 286 …
KTransformers is a GitHub project with over 17,000 stars that enables running massive 671-billion parameter models like DeepSeek-R1 on a single consumer machine (e.g., with 512GB RAM and one RTX 4090)…
KTransformers is an open-source inference library that enables running large language models with up to 671 billion parameters on a single consumer-grade machine by using heterogeneous placement to sp…