A homelab user deploying DeepSeek-V4-Flash-0731 on two DGX Spark units achieved 1M token context with an NVFP4 KV cache using a custom vLLM fork, reporting a KV cache size of 1,492,347 tokens and maxi…
A kernel is a single function that runs on a GPU, and the steep memory hierarchy—with a 1,600x gap between L2 cache and HBM—makes kernel design crucial for AI performance. FlashAttention, developed by…
The KV cache has no standard ABI, with vLLM's FlashAttention backend alone reporting its cache shape as a four-dimensional tensor that varies by backend, attention variant, and model family, complicat…
A base-2 vs. base-e log-sum-exp mismatch in SGLang's merge v2 API silently corrupts attention outputs for models using MLA with the FlashInfer backend when radix cache hits exceed 8k, affecting DeepSe…
A developer detailed deployment recipes for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter Mixture-of-Experts model with 95B active parameters, using vLLM. The guide covers verified GPU configurations, i…
Moonshot AI released the weights for Kimi K3 on July 27, a 2.8-trillion-parameter Mixture-of-Experts model with a 1-million-token context window, scoring third globally behind Claude Fable 5 Max and G…
LingBot-Map, a feed-forward 3D foundation model for streaming scene reconstruction, achieves state-of-the-art performance at approximately 20 FPS on 518×378 resolution over sequences exceeding 10,000 …
SGLang development is being augmented with agent-assisted workflows that encode procedural engineering knowledge into executable skills, covering LLM serving, GPU kernels, diffusion pipelines, and pro…
NVIDIA announced that its Confidential Computing technology for Blackwell GPUs achieves up to 98% of the inference performance of non-secure solutions, enabling hardware-rooted AI security without sig…
A developer reports that running gpt-oss-120B on an NVIDIA DGX Spark (GB10) failed due to MXFP4 quantization lacking native hardware support on SM121, causing kernel hangs and memory spikes. Switching…
Researchers at the FlashInfer MLSYS Challenge developed a formal verification method using the SPIN model checker to detect deadlocks in CuTe DSL kernels running on NVIDIA B200 GPUs. The approach, dem…
A new open-source tool, Auto GPU Kernel, autonomously discovers and optimizes GPU kernels, achieving a 34.93x average speedup to rank first in the DeepSeek Sparse Attention track of the MLSys 2026 Fla…