NVIDIA released CCCL Runtime, a modern C++ runtime for CUDA, as part of CUDA 13.2. The new APIs provide safer and more convenient abstractions for stream management, memory allocation, and kernel laun…
VLLM, a large-model inference serving framework, uses Python for control flow but pushes arithmetic into compiled C++ and CUDA kernels to avoid interpreter overhead. The Python/C++ boundary crossing i…
Google's TPU uses a systolic array architecture optimized for tensor algebra, offering higher throughput and energy efficiency than GPUs for dense matrix operations, but requires XLA compilation and i…
The open-source NVK Vulkan driver for Nvidia GPUs has merged experimental support for DLSS by importing pre-compiled CUDA binaries via the VK_NVX_binary_import extension, bypassing Nvidia's proprietar…
NVIDIA developer Norbert Juffa contributed a CUDA C++ implementation of the Haversine formula for fast great-circle distance calculations, leveraging the sinpi() and cospi() functions for improved per…
A developer built Omni-VRAM, an open-source voice AI platform with 28 modules. The platform includes speech recognition with five Whisper backends, real-time streaming under 200ms latency, speaker dia…
Egoist Machines, Inc. launched LodeDB, an open-source embedded vector database for local retrieval-augmented generation (RAG) that runs the same on-disk index on GPU when available, achieving up to 50…
Sasha Rush released GPU Puzzles, an interactive notebook teaching beginner GPU programming through coding exercises that map Python to CUDA kernels using NUMBA. The puzzles aim to build intuition for …
A developer released NanoEuler, a GPT-2-scale language model built entirely from scratch in C/CUDA without any machine learning libraries. The project includes a hand-written byte-level BPE tokenizer,…
A developer building vLLM from source on an AWS g5 instance with Ubuntu 26.04 and Python 3.14 encountered multiple version-skew, driver, and toolchain issues, including a pitfall where missing nvidia-…
Docker solves the 'it works on my machine' problem for AI engineers by packaging all dependencies—OS, Python, packages, code, and config—into a single portable image. A typical AI project has multiple…
Researchers extended the Heterogeneous Accelerator Toolkit (HAT) with a tensor-aware API and code transformations using OpenJDK Project Babylon's code reflection API, enabling Java programs to exploit…
Build b9437 of llama.cpp fixes two default-value bugs in llama-bench that caused flash attention to be skipped on capable GPUs and GPU-layer count to use a legacy sentinel. The flash attention flag no…
Loop unrolling, a classic compiler optimization, is experiencing a resurgence in the machine learning era as a critical technique for maximizing throughput on modern compute architectures like SIMD ve…
NVIDIA Labs released cuTile Rust, a tile-based DSL that extends Rust's ownership model to GPU programming, enabling memory-safe, data-race-free kernels without performance loss. Benchmarks on the B200…
Amazon's custom Trainium and Inferentia chips are gaining traction as organizations seek cost-effective alternatives to Nvidia GPUs for AI workloads. AWS reports cost reductions of 80-90% for inferenc…
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
Bernstein raised its price target on Advanced Micro Devices (AMD) to $600 from $525, citing strong momentum in the AI chip market. Analyst Stacy Rasgon maintained an Outperform rating, projecting fisc…
A developer calculated the actual VRAM requirements for running Llama 3 8B and Gemma 2 9B locally, revealing that the KV cache can consume far more memory than the model weights, especially at longer …
A developer achieved 2678x faster matrix multiplication using a GPU with CUDA, demonstrating how parallel processing on thousands of GPU cores reduces the O(N³) complexity of sequential matrix multipl…