Ways to improve my tokens / second

A practitioner running training on an RTX 5080 16GB with PyTorch 2.14 and CUDA 13 reported that torch.compile caused graph breaks and ran slightly slower than eager mode in a WSL environment with custom checkpointing, and that Liger's fused linear cross entropy failed with CUBLAS_STATUS_INTERNAL_ERROR and illegal memory access errors even on the nightly build. Reducing gradient checkpointing was not possible because baseline VRAM was already at 15.0 of 16.0 GB, so all 24 layers stayed checkpointed. The user reverted to native F.cross_entropy, which ran stably at roughly 9,000 tokens/s at peak VRAM.

Thank you for such a comprehensive and detailed breakdown I really appreciate the deep dive. I just finished running some A/B tests based on your suggestions, and here are the actual results on my setup RTX 5080 16GB, PyTorch 2.14, CUDA 13 : torch.compile : I tested eager vs. mode="default" . In my WSL environment with custom checkpointing, it caused graph breaks and was actually slightly slower than eager mode. Gradient Checkpointing : I tried reducing the checkpointed layers, but since my baseline VRAM is already at 15.0/16.0 GB, any reduction immediately caused an OOM Out of Memory . So I had to keep all 24 layers checkpointed. Fused Loss Liger : I tried to implement Liger’s fused linear cross entropy. Unfortunately, PyTorch 2.14 + CUDA 13 seems to have a deep compatibility issue with Liger. It threw CUBLAS STATUS INTERNAL ERROR and illegal memory access errors, even in the nightly build. Current Status : I reverted to the native F.cross entropy and it’s running perfectly stable at ~9,000 tokens/s with peak VRAM. Thank you again for the thorough guide It helped me rule out all the dead ends and find the most stable configuration for my hardware.