A developer investigating a vLLM crash on workstation Blackwell GPUs found that FP8 KV cache kernels exceed the per-block shared-memory limit on sm_120 hardware, causing OutOfResources errors. The iss…
An engineer benchmarked Qwen3-8B on workstation Blackwell GPUs across three serving stacks, finding that vLLM outperformed SGLang and llama.cpp by up to 4x under load, and that an FP8 pass delivered a…
Researchers introduced Self-Geometry, a test-time adaptation method that enforces explicit epipolar consistency to improve vision foundation models' depth and pose predictions without full retraining.…
FreeToken, a new inference engine from FlashML, lets users run frontier models on gaming GPUs at interactive speeds, with Qwen3.6 35B running on an 8GB RTX 4060 laptop at 39 tokens per second, DeepSee…
Runware, the AI inference startup founded by Flaviu Radulescu and Ioana Hreninciuc, launched Sonic Inference Pods, modular data centers in 20-foot shipping containers that pack about 1,200 GPUs and on…
Fermisense, an AI consultancy, reports that a 9B Qwen specialist fine-tuned for $500 on two RTX PRO 6000 GPUs over three and a half days scored 87.3% on a simulated catalog-review benchmark, outperfor…
Nvidia released Cosmos-H-Dreams, an interactive generative model that lets researchers operate a surgical robot inside a synthesized video environment, generating action-conditioned video rollouts at …
CamelAI self-hosts DeepSeek V4 Flash on AWS spot instances to power its free tier, using g7e.24xlarge instances with four RTX PRO 6000 GPUs. The setup keeps free-tier costs fixed by avoiding per-token…
James O'Beirne's GitHub repo 'local-llm' documents his experience building and running large language models on local hardware, covering configurations from a $2k setup with two RTX 3090s to a $40k bu…
Fine-tuning a 70B parameter model like LLaMA 3.1 locally requires significant VRAM, with a full fine-tune needing roughly 1.1 TB before activations and realistically 8× H100/A100-80GB GPUs with ZeRO-3…
VRAM capacity and memory bandwidth, not AI TOPS, are the decisive specs for local LLM inference in mid-2026, according to an analysis by Ji-ho Choi. NVIDIA's RTX 5090 leads with 32 GB VRAM and 1792 GB…
ARCQuant, a new framework for Large Language Model inference, uses the NVFP4 numerical format to achieve up to 3x speedup on GPUs while maintaining accuracy comparable to full-precision baselines. The…
NVIDIA announced that its Confidential Computing technology for Blackwell GPUs achieves up to 98% of the inference performance of non-secure solutions, enabling hardware-rooted AI security without sig…
NVIDIA introduced BEVPoolV3, an optimized bird's-eye-view pooling operator for autonomous vehicles and robotics, reducing latency on RTX GPUs by removing redundant scatter traffic and precomputing ind…
A 744-billion-parameter GLM-5.2 model was served at ~30 tokens per second across six prosumer Blackwell GPUs in six US states over a wide-area network using pipeline parallelism and speculative decodi…
Nvidia, in partnership with Span and PulteGroup, launched the XFRA initiative to install mini AI data centers in residential backyards, using surplus household electricity to power compact computing n…