Jane Street ML Infra intern Arsh Koneru built an activation checkpointing scheme that, given a model spec and a memory budget, selects a set of nodes to drop from the save set to minimize extra comput…
PyTorch 2.14 shipped on September 15 with 2,995 commits from 487 contributors, adding fault-tolerant distributed training as a first-class framework concept via a rewritten NCCL backend (nccl2) with i…
PyTorch 2.13, released this week, brings FlexAttention support for Apple Silicon with up to 12x speedup on sparse patterns and a new fused LinearCrossEntropyLoss operator that cuts peak GPU memory by …
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
PyTorch's Inductor compiler uses kernel fusion to accelerate model execution by up to 10x, grouping dependent operations into single Triton kernels to reduce memory traffic and kernel launch overhead.…