Astral has launched pre-built GPU-enabled Python wheels for the PyTorch ecosystem, supporting packages like FlashAttention across multiple CUDA versions (11.8 through 13.2) and PyTorch versions. The A…
Host overhead, caused by the CPU blocking the GPU, is a major source of inefficiency in AI inference, leading to low GPU kernel utilization and doubling GPU costs when at 50%. Modal recommends using t…
Tilery-VM, a virtual machine for NVIDIA's CUDA Tile IR bytecode, enables developers to run cuTile GPU kernels on a CPU without requiring a GPU or CUDA setup. The open-source project, hosted on GitHub,…
Researchers at UC Berkeley Sky Lab extended the K-Search evolutionary kernel optimization framework with a structured CUDA-to-MLX translation layer, enabling automatic transfer of decades of CUDA kern…
Open source remains a critical strategy for tech companies in the AI era, argues a former Borland employee drawing on lessons from Borland's failed open-source move with InterBase. Nvidia leverages it…
NERVOSYS founder Adam Erickson launched IronAccelerator on July 28th, a Rust-based CUDA interface that claims faster host-side performance than cudarc while preserving its API. NERVOSYS reports host-t…
A developer achieved up to 9.27× speedup on code editing tasks using adaptive speculative decoding on a €300 RTX 5060 GPU, with a 0.6B draft model nearly doubling math and JSON throughput. The project…
Eight months ago, software engineer Malte Skarupke wrote roughly 90% human code and 10% AI code; now his code is about 90% AI-generated. In a recent optimization of a matrix-multiply kernel using Nvid…
PrismML released the 1-bit Bonsai-27B language model, deployable via a specialized fork of llama.cpp with CUDA kernels for the Q1_0_g128 GGUF format. The model requires only ~5.2 GB peak memory at 4K …
Fermion Research has published Neutrino-1 8B, an 8.19-billion-parameter language model packaged in a 3.88 GB file that fits on an 8 GB GPU or 16 GB laptop, using a proprietary ternary-family format th…
LOCKS, a new method from arXiv, enables efficient long-context decoding by giving each page of the key-value cache its own compact spectral summary, reconstructing within-page logits, and attending on…
China's $295 billion AI infrastructure plan, announced in June 2026, aims to build a nationwide network of AI data centers with 80% domestic technology by 2028, sidestepping US chip export controls. T…
A developer deploying the Kimi K3 model encountered a CUDA out-of-memory error caused by KV cache allocation during initial inference passes, not the model weights themselves. The developer resolved t…
The 5th edition of 'Programming Massively Parallel Processors' by Wen-mei W. Hwu, David B. Kirk, and Izzat El Hajj, published by Morgan Kaufmann, introduces new chapters on filtering, wavefront parall…
A developer reports that deploying Google's Gemma model causes a CUDA out-of-memory error during the weights loading sequence, with 18.1GB already allocated on a 24GB GPU. The spike occurs during the …
A self-study AI engineer demonstrates how to fit a large language model on a 6GB laptop GPU using quantization, a technique that reduces model precision to enable fine-tuning on consumer hardware. The…
AMD has published a machine-readable ISA for its Instinct GPUs and partnered with Anthropic and OpenAI to let frontier models natively write and optimize low-level kernels, claiming a 38% inference sp…
Nvidia CEO Jensen Huang now supports open source, raising expectations for CUDA to be open-sourced, according to a post on X.…
AMD's move to provide machine-readable Instruction Set Architecture (ISA) specifications could transform GPU programming by enabling LLM agents to directly generate optimized kernels, bypassing the ne…
Nvprobe, an open-source, zero-setup CLI tool for CUDA benchmarks, has been released on GitHub. The tool automates CUDA workloads including HPL, HPCG, MLPerf inference, and custom kernels, and generate…