A developer achieved up to 9.27× speedup on code editing tasks using adaptive speculative decoding on a €300 RTX 5060 GPU, with a 0.6B draft model nearly doubling math and JSON throughput. The project…
Eight months ago, software engineer Malte Skarupke wrote roughly 90% human code and 10% AI code; now his code is about 90% AI-generated. In a recent optimization of a matrix-multiply kernel using Nvid…
PrismML released the 1-bit Bonsai-27B language model, deployable via a specialized fork of llama.cpp with CUDA kernels for the Q1_0_g128 GGUF format. The model requires only ~5.2 GB peak memory at 4K …
Fermion Research has published Neutrino-1 8B, an 8.19-billion-parameter language model packaged in a 3.88 GB file that fits on an 8 GB GPU or 16 GB laptop, using a proprietary ternary-family format th…
LOCKS, a new method from arXiv, enables efficient long-context decoding by giving each page of the key-value cache its own compact spectral summary, reconstructing within-page logits, and attending on…
China's $295 billion AI infrastructure plan, announced in June 2026, aims to build a nationwide network of AI data centers with 80% domestic technology by 2028, sidestepping US chip export controls. T…
A developer deploying the Kimi K3 model encountered a CUDA out-of-memory error caused by KV cache allocation during initial inference passes, not the model weights themselves. The developer resolved t…
The 5th edition of 'Programming Massively Parallel Processors' by Wen-mei W. Hwu, David B. Kirk, and Izzat El Hajj, published by Morgan Kaufmann, introduces new chapters on filtering, wavefront parall…
A developer reports that deploying Google's Gemma model causes a CUDA out-of-memory error during the weights loading sequence, with 18.1GB already allocated on a 24GB GPU. The spike occurs during the …
A self-study AI engineer demonstrates how to fit a large language model on a 6GB laptop GPU using quantization, a technique that reduces model precision to enable fine-tuning on consumer hardware. The…
AMD has published a machine-readable ISA for its Instinct GPUs and partnered with Anthropic and OpenAI to let frontier models natively write and optimize low-level kernels, claiming a 38% inference sp…
Nvidia CEO Jensen Huang now supports open source, raising expectations for CUDA to be open-sourced, according to a post on X.…
AMD's move to provide machine-readable Instruction Set Architecture (ISA) specifications could transform GPU programming by enabling LLM agents to directly generate optimized kernels, bypassing the ne…
Nvprobe, an open-source, zero-setup CLI tool for CUDA benchmarks, has been released on GitHub. The tool automates CUDA workloads including HPL, HPCG, MLPerf inference, and custom kernels, and generate…
TileLang, a high-level Python domain-specific language for designing GPU kernels through TVM, enables developers to implement tensor-core GEMM, fused softmax, FlashAttention, and autotuning while mana…
A machine learning intern recounts that the biggest challenge in transitioning from a local environment to deployment was dependency conflicts, specifically a PyTorch version mismatch with CUDA driver…
Anthropic researcher Julian Schrittwieser mocked NVIDIA and Microsoft for signing an open AI letter, calling on them to open-source CUDA and Microsoft Office. Schrittwieser highlighted the irony of hi…
Inflect v2, a text-to-speech model under 10 million parameters, achieves competitive performance with 4.395 UTMOS22 and 3.99% semantic WER for the Micro version (9.36M parameters) and 4.386 UTMOS22 wi…
AMD at its Advancing AI event in San Francisco unveiled ROCm.AI, a platform that uses frontier AI models to automatically optimize GPU kernels and inference performance on AMD Instinct hardware, claim…
NVIDIA CEO Jensen Huang's new X account signals a shift toward real-time, decentralized communication with the developer community, potentially accelerating feedback loops on AI hardware and software …