A technical guide from Cast AI explains that quantization, the process of compressing LLM weights to lower-precision data types, is central to balancing throughput, memory, and inference costs, and cl…
A developer's hand-written Gemma 4 port in pure JAX runs on both Turing and Ada NVIDIA GPUs with identical weights, but performance differs drastically due to hardware-specific compute dtype handling …
A developer explores Triton, a Python-based language and compiler for writing GPU kernels, to understand low-level GPU programming and performance. The developer explains how Triton abstracts GPU thre…
A developer's hand-written Gemma 4 port in pure JAX runs across TPU v5e/v6e and NVIDIA T4G GPUs, revealing that while most of the code is portable, two hardware-specific issues break the abstraction. …
PyTorch Conference North America 2026, scheduled for October 20–21 in San Jose, CA, will feature Core PyTorch sessions covering compiler and runtime internals, distributed communication, device portab…
A new benchmark, DataKernelBench, evaluates whether large language models can optimize database queries on GPUs, achieving up to 2.11x speedup over torch.compile on TPC-H SF10 with an H100 GPU. The be…
ClearML, an open-source MLOps platform, can be deployed as a self-hosted alternative to Azure Machine Learning, offering experiment tracking, pipelines, and model serving on any infrastructure. A deve…
A developer investigating a vLLM crash on workstation Blackwell GPUs found that FP8 KV cache kernels exceed the per-block shared-memory limit on sm_120 hardware, causing OutOfResources errors. The iss…
Higgsfield AI, a generative AI company with $500M in annual revenue run rate and 25M+ users, is hiring an ML Systems Performance Engineer (MFU) for its Almaty, Kazakhstan office. The role focuses on o…
Researchers introduced HAWKEYE, an open-source framework that uses only 10 expert-written unit tests per GPU architecture to enable AI coding agents to generate hardware-aware kernels, approaching or …
Pine AI's 1.2B-parameter speech recognition model tops the τ³-Voice leaderboard with a score of 75, achieving significant gains on accented speech, medical dictation, and code-switching, but only marg…
A kernel is a single function that runs on a GPU, and the steep memory hierarchy—with a 1,600x gap between L2 cache and HBM—makes kernel design crucial for AI performance. FlashAttention, developed by…
A Kubernetes engineer's guide compares LLM serving engines, arguing that vLLM is not the only option and that choosing an engine is a critical production decision. The post highlights vLLM's strengths…
The KV cache has no standard ABI, with vLLM's FlashAttention backend alone reporting its cache shape as a four-dimensional tensor that varies by backend, attention variant, and model family, complicat…
A new arXiv preprint (2608.13756v1) reports that swapping only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM causes the two arms to agree on no sequence in any end-to-end comparison (0/8,…
Triton 3.7's new plugin extension system lets GPU kernel developers load custom MLIR compiler passes at runtime as shared libraries, eliminating the need to maintain a Triton fork. Meta's TLX extensio…
A solo developer with no professional GPU background placed 12th of 183 in GPU MODE's batched QR decomposition contest, beating the cuSolver-backed baseline by 232x (419,000 µs down to 1,805 µs on NVI…
A new study comparing CUDA C++, Rust via NVIDIA's cuda-oxide, and Triton on a hash-blocked TSDF fusion kernel found that Triton is more than an order of magnitude slower than hand-written CUDA C++ on …
A developer has open-sourced a Decay-Gated O(N) Causal Linear Attention architecture with fused Triton/CUDA kernels, aiming to bypass quadratic multi-head attention bottlenecks. The project includes a…
The US lead over China in AI has essentially disappeared, according to an analysis of model releases and deployment trends. Chinese models like DeepSeek's R1 and V3 and Qwen now match or beat US open-…