KTransformers 0.7 Expands AVX-512 Support to Benefit AMD EPYC Servers

KTransformers 0.7, the heterogeneous LLM inference and fine-tuning framework, is now available with full AVX-512 support for LoRA fine-tuning that no longer requires Advanced Matrix Extensions (AMX), benefiting AMD EPYC Zen 4, Zen 5, and Zen 6 servers and older Intel Xeon processors. The release also adds VLM fine-tuning support, native FP8 LoRA support, improved DeepSeek V4 deployment, and CPU activation reuse.

KTransformers 0.7 Expands AVX-512 Support To Benefit AMD EPYC Servers KTransformers as the framework for heterogeneous LLM inference and fine-tune optimizations is out today with its v0.7 feature release. With KTransformers 0.7 there is now full The KTransformers 0.7 also adds VLM fine-tuning support, native FP8 LoRA support, improved DeepSeek V4 deployment, and CPU activation reuse. More details on KTransformers 0.7 for those using it for LLM inference optimizations and fine-tuning can find all the details via the release announcement on With KTransformers 0.7 there is now full AVX-512 https://www.phoronix.com/search/AVX-512 support for LoRA fine-tuning without depending upon Advanced Matrix Extensions AMX https://www.phoronix.com/search/AMX also being present. This AVX-512-only without AMX benefits AMD EPYC Zen 4 / Zen 5 / Zen 6 servers with excellent AVX-512 support while lacking AMX and also older Intel Xeon processors with AVX-512 prior to the introduction of AMX with Sapphire Rapids.The merge request https://github.com/kvcache-ai/ktransformers/pull/2141 noted the testing on AMD hardware and the foxus on AVX-512 without AMX platforms. The KTransformers runtime will automatically select the proper CPU implementation and in turn allowing MoE expert training to happen on a wider range of large-memory servers.KTransformers 0.7 also adds VLM fine-tuning support, native FP8 LoRA support, improved DeepSeek V4 deployment, and CPU activation reuse. More details on KTransformers 0.7 for those using it for LLM inference optimizations and fine-tuning can find all the details via the release announcement on