A developer published a Dockerfile that layers Aleph Alpha's proprietary Kolibri-1 inference plugin onto a native SM121 (GB10) vLLM build for the DGX Spark, preserving the Blackwell-compiled kernels. …
PTXBench, a benchmark released August 17, 2026 with code on GitHub, evaluates how well frontier LLMs can write architecture-specific CUDA-PTX for H100 and B200 GPUs, finding GEMM nearly solved while a…
Perplexity Engineering detailed its GPU embedding serving stack for pplx-embed, revealing that it reuses prefill and decode kernels from its LLM engine rather than building a separate embedding engine…
A technical guide details the production deployment of on-premises large language model inference using vLLM, covering an 8-GPU node running GLM-5.2/5.3 (NVFP4 MoE) and a single L40S running gemma-4-2…
A bug is a violation of a specification, and no specification exists that prefix caching's variable logits violate, according to an analysis of vLLM and SGLang issues. The vLLM PR #34046 adds an opt-i…
A homelab user deploying DeepSeek-V4-Flash-0731 on two DGX Spark units achieved 1M token context with an NVFP4 KV cache using a custom vLLM fork, reporting a KV cache size of 1,492,347 tokens and maxi…
A kernel is a single function that runs on a GPU, and the steep memory hierarchy—with a 1,600x gap between L2 cache and HBM—makes kernel design crucial for AI performance. FlashAttention, developed by…
The KV cache has no standard ABI, with vLLM's FlashAttention backend alone reporting its cache shape as a four-dimensional tensor that varies by backend, attention variant, and model family, complicat…
A base-2 vs. base-e log-sum-exp mismatch in SGLang's merge v2 API silently corrupts attention outputs for models using MLA with the FlashInfer backend when radix cache hits exceed 8k, affecting DeepSe…
A developer detailed deployment recipes for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter Mixture-of-Experts model with 95B active parameters, using vLLM. The guide covers verified GPU configurations, i…
Moonshot AI released the weights for Kimi K3 on July 27, a 2.8-trillion-parameter Mixture-of-Experts model with a 1-million-token context window, scoring third globally behind Claude Fable 5 Max and G…
LingBot-Map, a feed-forward 3D foundation model for streaming scene reconstruction, achieves state-of-the-art performance at approximately 20 FPS on 518×378 resolution over sequences exceeding 10,000 …
SGLang development is being augmented with agent-assisted workflows that encode procedural engineering knowledge into executable skills, covering LLM serving, GPU kernels, diffusion pipelines, and pro…
NVIDIA announced that its Confidential Computing technology for Blackwell GPUs achieves up to 98% of the inference performance of non-secure solutions, enabling hardware-rooted AI security without sig…
A developer reports that running gpt-oss-120B on an NVIDIA DGX Spark (GB10) failed due to MXFP4 quantization lacking native hardware support on SM121, causing kernel hangs and memory spikes. Switching…
Researchers at the FlashInfer MLSYS Challenge developed a formal verification method using the SPIN model checker to detect deadlocks in CuTe DSL kernels running on NVIDIA B200 GPUs. The approach, dem…
A new open-source tool, Auto GPU Kernel, autonomously discovers and optimizes GPU kernels, achieving a 34.93x average speedup to rank first in the DeepSeek Sparse Attention track of the MLSys 2026 Fla…