LLaMA-Factory

A first-principles analysis of LoRA fine-tuning memory usage reveals that 87.3% of VRAM scaling with sequence length for Llama 3.1 8B is consumed by the cross-entropy loss head tensor, not the model, …

KTransformers, a Tsinghua-born framework now with 19k GitHub stars, has restructured from a standalone inference system into CPU kernels and placement strategies embedded within SGLang and LLaMA-Facto…

Unsloth achieves up to 7.3x training speedup over standard Transformers for MoE models like gpt-oss-20B on an NVIDIA B200, according to published benchmarks, while Axolotl delivers up to 1.45x speedup…