A new book, 'Modern GPU Programming For MLSys', teaches GPU kernel optimization for machine learning systems, focusing on Blackwell architecture and techniques like GEMM and FlashAttention. Developed …
NVIDIA released a guide showing how to optimize transformer-based models for low-precision training using Hopper and Blackwell GPUs, focusing on FP8 and NVFP4 formats. The method translates model conf…
Researchers at an undisclosed institution reverse-engineered Apple's Metal 4.1 tensor compute path on the M4 Max GPU, revealing that the fp8 matmul2d operation is emulated rather than hardware-acceler…
CODA, a GPU kernel abstraction that reparameterizes memory-bound Transformer operations like normalization and activations to execute as GEMM-plus-epilogue programs, keeping data on-chip to reduce glo…