Researchers have developed a compiler-grounded hierarchical optimization framework for Triton kernels that achieves a geometric-mean speedup of 4.35× and a median speedup of 2.73× across 37 benchmark …
At the AMD Advancing AI event, analysts from theCUBE Research highlighted AMD's transformation into a full-stack AI infrastructure company, with Dave Vellante noting AMD's $60 billion in M&A, includin…
JAX's Pallas kernel system, which lowers through Triton on GPU and Mosaic on TPU, lets developers write custom kernels when XLA's automatic fusion is insufficient for operations like flash attention, …
Netflix detailed its in-house LLM serving platform built on Triton and vLLM, revealing how it handles real-time and batch inference across CPUs and GPUs while managing version compatibility and constr…
NVIDIA CEO Jensen Huang's new X account signals a shift toward real-time, decentralized communication with the developer community, potentially accelerating feedback loops on AI hardware and software …
SonicSampler, a unified suite of tile-aware Triton kernels from arXiv, vertically fuses the complete LLM sampling pipeline into a fixed, workload-aware execution model, achieving up to 16x speedup ove…
A developer building the Triton DirectX 11 driver for QEMU argues that AI tools amplify human developers rather than replace them, citing challenges such as sparse documentation, multi-boundary debugg…
Triton-blackhole, a deterministic numerical debugger for Triton kernels, identifies where output diverges from a PyTorch reference and classifies drift as benign fp16/bf16 drift or a real bug without …
Stanford NLP's CS336 course, audited by a student who built an LLM from scratch for $353, delivers on its premise of deepening understanding of modern LLMs through hands-on assignments that include bu…
A developer's practical guide details the engineering challenges and solutions for deploying production-ready Retrieval-Augmented Generation (RAG) applications, covering data indexing, vector stores, …
Torch.compile is not a traditional compiler but a system that intercepts Python bytecode at runtime, extracts compilable regions through a multi-stage pipeline, and stitches them back with eager Pytho…
A developer compares PyTorch and TensorFlow for computer vision projects, noting that PyTorch dominates research with dynamic computation graphs and recent performance improvements via torch.compile()…
Triton is a compiler with a Python frontend that parses a function's AST, runs it through an MLIR pipeline, and emits a GPU binary, never executing the Python function as Python. The compiler handles …
A 200-step, ~1.7B-parameter ablation run in OLMo-core comparing Engram on a standard attention Transformer versus a 3 GDN-layer + 1 attention-layer hybrid found that Transformer + Engram reached sligh…
The PyTorch-Triton 3.7 release introduces the Triton Plugin Extensions system, a framework for dynamically loading custom compiler passes, dialects, and DSL extensions into upstream Triton at runtime …
Unsloth AI Founding Engineer Challenge #1 produced a Triton GPU kernel for NF4 dequantization that achieves 1.27x–1.72x speedup over the existing bitsandbytes C++ implementation across all tested tens…
AMD has released a Triton-based optimization for video sparse attention on its ROCm platform, targeting Diffusion Transformers (DiTs) used in video generation. The implementation reduces the quadratic…
AMD's open-source GEAK agent-driven framework automated the optimization of the DeepSeekV4 MLA kernel, achieving a 2.10x improvement in end-to-end throughput and a 3.71x reduction in time-to-first-tok…
Hugging Face announced that custom GPU kernels on its Hub are now signed artifacts governed by a trusted publisher model, requiring a dedicated repository type that replaces the old model-type format.…
Gimlet Labs has built an early research system that uses formal verification to prove semantic equivalence between reference PyTorch models and AI-generated GPU kernels, catching bugs that pass tradit…