Anthropic member of technical staff Yifan Yang reported that a CUDA-core fp32 GEMM kernel on an Nvidia V100 GPU reached 10.7 TFLOPS, or 88% of cuBLAS's 12.1 TFLOPS, using only two optimizations over a…
A developer released rGPU, an open-source project that lets PyTorch programs run GPU work on a remote NVIDIA machine via a custom `rgpu` device or a CUDA shim covering libcuda, CUDA Runtime, cuBLAS, c…
A technical case study reports that a single NVIDIA RTX 4090 with 24 GB of VRAM can run a 125-billion-parameter Qwen 3.8 Flash Next model at roughly 100 trillion tokens per second, a throughput the wr…
PTXBench, a benchmark released August 17, 2026 with code on GitHub, evaluates how well frontier LLMs can write architecture-specific CUDA-PTX for H100 and B200 GPUs, finding GEMM nearly solved while a…
OpenGEMM, an open-source project from developer aramesh10, released CUDA GEMM kernels for NVIDIA's B200 GPU (sm_100a), installable via pip and requiring PyTorch 2.8+ and CUDA 12.9+. The library emits …
SGLang and vLLM, the two major LLM serving engines, have attempted to enforce deterministic inference but face fundamental challenges due to floating-point non-associativity in GEMM kernels, which cau…
A bug is a violation of a specification, and no specification exists that prefix caching's variable logits violate, according to an analysis of vLLM and SGLang issues. The vLLM PR #34046 adds an opt-i…
A developer's optimization campaign on an RTX 3080 Ti cut a GPT-2-small-class transformer's forward pass from 78.2ms to 1.60ms, a 49× speedup, beating torch.compile's 1.72ms and reaching 136,000 token…
TileLang, a high-level Python domain-specific language for designing GPU kernels through TVM, enables developers to implement tensor-core GEMM, fused softmax, FlashAttention, and autotuning while mana…
Torch.compile is not a traditional compiler but a system that intercepts Python bytecode at runtime, extracts compilable regions through a multi-stage pipeline, and stitches them back with eager Pytho…
TornadoVM now natively integrates NVIDIA CUDA libraries cuBLAS, cuFFT, and cuDNN, allowing Java applications to call GPU-accelerated linear algebra and deep learning primitives directly from JIT-compi…
TornadoVM, a Java heterogeneous programming framework, now supports CUDA Graphs, multiple streams, and vendor libraries like cuBLAS, cuFFT, and cuDNN, enabling Java applications to leverage NVIDIA's t…
A developer built Phobos, a tiny scale-free kernel language that compiles to PTX and runs on NVIDIA GPUs, achieving 76% of cuBLAS SGEMM performance on a 2080 SUPER. The language supports tile-DAG desi…
Developer Vincenzo released NanoEuler, a from-scratch GPT-2-scale language model training stack written in C and CUDA without PyTorch or autograd, on June 28, 2026. The 116-million-parameter project i…
A developer released NanoEuler, a GPT-2-scale language model built entirely from scratch in C/CUDA without any machine learning libraries. The project includes a hand-written byte-level BPE tokenizer,…
A developer open-sourced AutoMegakernel, a tool that compiles any HuggingFace model into a single persistent megakernel, reducing overhead by launching one kernel per forward pass. It includes a stati…
Researchers introduced cuTile Rust, a tile-based system for safe, idiomatic GPU kernel authoring in Rust that extends Rust's ownership discipline to GPU kernels. On the NVIDIA B200 GPU, cuTile Rust ac…
NVIDIA's CUTLASS library, a header-only C++ template framework for writing custom CUDA kernels, powers much of the AI infrastructure behind FlashAttention, vLLM, and PyTorch's internal kernels. The li…