Meta's LLMs have evolved from simple Transformer stacks to complex architectures with multiple attention variants, mixture-of-experts, and multimodal encoders, mirroring the complexity of recommendati…
OpenTelemetry's graduation to stable status within the CNCF provides a standardized observability framework for production LLM pipelines, addressing challenges in tracing inference requests across vec…
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
CyCognito launched Kineto, a continuous AI pentesting capability that simulates multi-step attack chains across enterprise infrastructure. The platform uses AI agents to evaluate sequences of actions …
NVIDIA researchers developed an agentic system for deploying machine learning models to ephemeral SageMaker endpoints, generating runtime code at deployment time from prose artifacts rather than reusa…
Polar Signals has released an open-source, low-overhead continuous profiler for NVIDIA CUDA that supports Program Counter (PC) sampling, allowing developers to see where GPU code spends time at the in…
Researchers from UC Berkeley and UT Austin released Flash-KMeans, an IO-aware, exact k-means library that runs over 200× faster than FAISS on GPUs by restructuring data movement. The open-source libra…
Researchers developed custom Triton kernels that accelerate JumpReLU Sparse Autoencoder inference by 2–14× on real SAEs, exploiting activation sparsity to skip zero entries during matrix multiplicatio…
NVIDIA's CUTLASS library, a header-only C++ template framework for writing custom CUDA kernels, powers much of the AI infrastructure behind FlashAttention, vLLM, and PyTorch's internal kernels. The li…
PyTorch's Inductor compiler uses kernel fusion to accelerate model execution by up to 10x, grouping dependent operations into single Triton kernels to reduce memory traffic and kernel launch overhead.…
NVIDIA's C++ template-based CUTLASS library for GPU kernels suffers from compile times of up to 20 seconds for a single kernel and over 17 minutes for full builds, prompting a shift toward Python-embe…
NVIDIA released CompileIQ, an AI-powered compiler auto-tuning framework that uses evolutionary and genetic algorithms to optimize GPU compilers for individual workloads. The tool, included in NVIDIA C…
TileLang introduces a middle-ground approach for writing high-performance GPU kernels, offering explicit control over shared memory allocation, pipeline staging, and warp partitioning through Python c…
PyTorch has introduced transparent tracing and compilation for Triton kernels, allowing custom operations to be visible to the compiler for optimization. The framework now supports compiling Triton ke…