KTransformers

The MADSys Lab at Tsinghua University’s KTransformers project enables frontier-class MoE models like DeepSeek-R1 671B to run on commodity hardware with a CPU-GPU hybrid inference stack, achieving 286 …

KTransformers is a GitHub project with over 17,000 stars that enables running massive 671-billion parameter models like DeepSeek-R1 on a single consumer machine (e.g., with 512GB RAM and one RTX 4090)…

KTransformers is an open-source inference library that enables running large language models with up to 671 billion parameters on a single consumer-grade machine by using heterogeneous placement to sp…