PyTorch 2.13, released July 8, brings FlexAttention to Apple Silicon with up to 12x speedup on sparse attention patterns, such as a 32,768-token sequence with a 256-token sliding window (35ms vs 431ms…
A developer benchmarked PyTorch 2.13's FlexAttention on an M1 Max Mac, finding up to 7.83x speedup over standard SDPA for sparse attention with 32,768 tokens and a 256-token local window. FlexAttentio…
PyTorch 2.13, released this week, brings FlexAttention support for Apple Silicon with up to 12x speedup on sparse patterns and a new fused LinearCrossEntropyLoss operator that cuts peak GPU memory by …
Meta's LLMs have evolved from simple Transformer stacks to complex architectures with multiple attention variants, mixture-of-experts, and multimodal encoders, mirroring the complexity of recommendati…