Developer Vincenzo released NanoEuler, a from-scratch GPT-2-scale language model training stack written in C and CUDA without PyTorch or autograd, on June 28, 2026. The 116-million-parameter project i…
A new book, 'Modern GPU Programming For MLSys', teaches GPU kernel optimization for machine learning systems, focusing on Blackwell architecture and techniques like GEMM and FlashAttention. Developed …
A developer released NanoEuler, a GPT-2-scale language model built entirely from scratch in C/CUDA without any machine learning libraries. The project includes a hand-written byte-level BPE tokenizer,…
Miami startup Subquadratic claims to have solved a decade-old bottleneck in AI models by replacing dense attention with sparse attention, achieving 56x speed gains and dramatic cost reductions. Indepe…
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
A blog post derives the backward pass of attention mechanisms step by step, using a concrete example to illustrate gradient computation for Q, K, and V matrices. The derivation builds on FlashAttentio…
Shrijith Venkatramana, developer of git-lrc, explains how kernel fusion reduces memory bandwidth bottlenecks in LLM inference. By combining multiple GPU operations into a single kernel, intermediate d…
A new dataset, fineset-io/efficient-llm-papers, compiles 1,734 records of arXiv and Semantic Scholar papers on efficient LLM techniques like quantization, LoRA, MoE, and FlashAttention, each quality-s…
Rudrite Research launched a free, open platform offering interactive, animated visual explainers of landmark AI and ML papers, including Attention Is All You Need, GPT-3, and FlashAttention, to make f…
Rudrite Research published a comprehensive comparison of AI and ML approaches, covering 14 side-by-side analyses of techniques such as Transformers vs Mamba, FlashAttention vs PagedAttention, and PPO …
A developer has released tiny-vllm, a high-performance LLM inference engine written in C++ and CUDA that serves as a smaller sibling to the vLLM project. The open-source repository includes both the f…
NVIDIA's Blackwell consumer GPU (GB203/SM_120) features a unified TensorCore pipeline where all 12 non-FP64 precision formats share identical 29-cycle latency and 23-cycle throughput, reducing precisi…
NVIDIA's CUTLASS library, a header-only C++ template framework for writing custom CUDA kernels, powers much of the AI infrastructure behind FlashAttention, vLLM, and PyTorch's internal kernels. The li…
FlashLib, a new GPU library for classical machine learning operators, achieves speedups of up to 208× over cuML on Hopper GPUs for algorithms including KMeans, KNN, and PCA. The library is designed to…
A developer implemented FlashAttention's forward and backward passes from scratch in pure CUDA C++, achieving O(N) memory complexity through manual SRAM tiling and online softmax recurrence. A rejecte…
Synchronous continuous batching in LLM inference causes inefficiency by forcing the CPU and GPU to work sequentially, leaving one idle while the other operates. This idle time can account for nearly a…