A kernel is a single function that runs on a GPU, and the steep memory hierarchy—with a 1,600x gap between L2 cache and HBM—makes kernel design crucial for AI performance. FlashAttention, developed by…
A new arXiv preprint (2608.13756v1) reports that swapping only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM causes the two arms to agree on no sequence in any end-to-end comparison (0/8,…
An audit by computer science researchers finds that NVIDIA's Blackwell Ultra GPU (B300) nominally supports INT8 tensor-core compute at a 30:1 FP8-to-INT8 ratio, but the format is undeployable by defau…
NVIDIA researcher Cris Cecka released PyCuTe, a pure-Python reference implementation of the CuTe layout algebra used in CUTLASS 3.x and the CuTe DSL, enabling learning, prototyping, and test-vector ge…
PyTorch 2.13, released this week, brings FlexAttention support for Apple Silicon with up to 12x speedup on sparse patterns and a new fused LinearCrossEntropyLoss operator that cuts peak GPU memory by …
Researchers introduced two CUTLASS-based kernels that accelerate the SwiGLU activation function in large language models by up to 2.47x on NVIDIA H100 GPUs, shifting workloads from memory-bound to com…
INT21 has built SwarmOS, a cloud-native platform for multi-agent AI systems, arguing that orchestrating specialized agents is more effective than relying on larger context windows. The company demonst…
Helion kernels were integrated into vLLM for FP8 inference using Qwen3 models and evaluated across NVIDIA H100 and B200 GPUs. The experiments demonstrated that Helion provides a productive PyTorch-nat…
NVIDIA's Blackwell consumer GPU (GB203/SM_120) features a unified TensorCore pipeline where all 12 non-FP64 precision formats share identical 29-cycle latency and 23-cycle throughput, reducing precisi…
NVIDIA's CUTLASS library, a header-only C++ template framework for writing custom CUDA kernels, powers much of the AI infrastructure behind FlashAttention, vLLM, and PyTorch's internal kernels. The li…
Researchers at the FlashInfer MLSYS Challenge developed a formal verification method using the SPIN model checker to detect deadlocks in CuTe DSL kernels running on NVIDIA B200 GPUs. The approach, dem…
TileLang introduces a middle-ground approach for writing high-performance GPU kernels, offering explicit control over shared memory allocation, pipeline staging, and warp partitioning through Python c…
Matrix multiplications on Nvidia A100 GPUs run up to 15% faster when the input matrices contain predictable values like zeros or integers, rather than random data. The performance difference stems fro…