@DeepSpeed

Researchers at Microsoft have introduced AutoSP, a compiler-based solution that automatically converts standard training code into multi-GPU sequence parallel code for long-context language model trai…