Stanford NLP's CS336 course, audited by a student who built an LLM from scratch for $353, delivers on its premise of deepening understanding of modern LLMs through hands-on assignments that include bu…
A developer's practical guide details the engineering challenges and solutions for deploying production-ready Retrieval-Augmented Generation (RAG) applications, covering data indexing, vector stores, …
Torch.compile is not a traditional compiler but a system that intercepts Python bytecode at runtime, extracts compilable regions through a multi-stage pipeline, and stitches them back with eager Pytho…
A developer compares PyTorch and TensorFlow for computer vision projects, noting that PyTorch dominates research with dynamic computation graphs and recent performance improvements via torch.compile()…
Triton is a compiler with a Python frontend that parses a function's AST, runs it through an MLIR pipeline, and emits a GPU binary, never executing the Python function as Python. The compiler handles …
A 200-step, ~1.7B-parameter ablation run in OLMo-core comparing Engram on a standard attention Transformer versus a 3 GDN-layer + 1 attention-layer hybrid found that Transformer + Engram reached sligh…
The PyTorch-Triton 3.7 release introduces the Triton Plugin Extensions system, a framework for dynamically loading custom compiler passes, dialects, and DSL extensions into upstream Triton at runtime …
Unsloth AI Founding Engineer Challenge #1 produced a Triton GPU kernel for NF4 dequantization that achieves 1.27x–1.72x speedup over the existing bitsandbytes C++ implementation across all tested tens…
AMD has released a Triton-based optimization for video sparse attention on its ROCm platform, targeting Diffusion Transformers (DiTs) used in video generation. The implementation reduces the quadratic…
AMD's open-source GEAK agent-driven framework automated the optimization of the DeepSeekV4 MLA kernel, achieving a 2.10x improvement in end-to-end throughput and a 3.71x reduction in time-to-first-tok…
Hugging Face announced that custom GPU kernels on its Hub are now signed artifacts governed by a trusted publisher model, requiring a dedicated repository type that replaces the old model-type format.…
Gimlet Labs has built an early research system that uses formal verification to prove semantic equivalence between reference PyTorch models and AI-generated GPU kernels, catching bugs that pass tradit…
AMD contributed a compatibility update to JAX-Triton that supports most Triton features, enabling users to run virtually any Triton or Gluon kernel inside JAX with minimal changes. The update includes…
A developer built Phobos, a tiny scale-free kernel language that compiles to PTX and runs on NVIDIA GPUs, achieving 76% of cuBLAS SGEMM performance on a 2080 SUPER. The language supports tile-DAG desi…
Researchers introduce PuzzleMoE, a method for compressing large Mixture-of-Experts models via fine-grained element-wise merging and bit-packing, achieving up to 16.7% higher accuracy on MMLU at 50% co…
Stockholmsbörsen öppnar avvaktande efter en stark period på Wall Street, där teknikaktier drev Nasdaq upp 1,5 procent. SEB konstaterar att marknaden har sett igenom geopolitisk oro och satsat stort på…
AMD announced a new kernel family, LDS-Pipelined Split-K GEMM, that accelerates LLM inference on AMD GPUs by optimizing decode-time GEMMs with small M and large N/K dimensions. The technique achieves …
The OpenXLA compiler stack and JAX now run upstream on AMD ROCm, with XLA gating every pull request on real AMD Instinct silicon through GitHub Actions and JAX running hardware tests on every ROCm PR.…
MLIR, a compiler infrastructure framework, has become the foundation for numerous machine learning compilers including XLA, Triton, Mojo, Torch-MLIR, IREE, and ONNX-MLIR. It provides a reusable IR con…
A developer confirmed that GLM-5.2, a 753B-parameter DeepSeek-Sparse-Attention MoE model, runs on 8x A100 80GB GPUs using vLLM PR #38476, which adds a Triton sparse-MLA backend for Ampere architecture…