Fault tolerant distributed training on Amazon EKS using NVRx
NVIDIA's Resiliency Extension (NVRx) can be integrated into PyTorch Fully Sharded Data Parallel training on Amazon Elastic Kubernetes Service to cut idle GPU time, with synchronous checkpointing consuming up to 40% of to…