Nvidia's H200 processors have reached mainland China in limited quantities, with ByteDance and Tencent each receiving about 10,000 units in recent weeks, according to the Financial Times. U.S. regulat…
Groq is spending billions to poach Nvidia engineers as it targets the AI inference market, aiming to challenge Nvidia's dominance by optimizing hardware for the sequential nature of LLM token generati…
The memory ceiling on local GPUs is the most reliable predictor of cloud inference cost trends, according to an analysis of 2026 hardware and pricing data. The article calculates that a 70B model requ…
Groq has raised $350 million to expand its neocloud data center footprint, integrating Nvidia-powered clusters alongside its proprietary Language Processing Unit (LPU) technology to offer a hybrid env…
Renting Nvidia's H100 GPU climbed roughly 40% between October 2025 and March 2026, with average rates jumping from $1.70 to $2.35 per GPU-hour, according to data from SemiAnalysis. On-demand H100 capa…
Triton 3.7's new plugin extension system lets GPU kernel developers load custom MLIR compiler passes at runtime as shared libraries, eliminating the need to maintain a Triton fork. Meta's TLX extensio…
Cloudflare released Unweight, a lossless compression system that reduces BF16 MLP weights in LLMs by 15–22% while producing bit-identical outputs, freeing roughly 3 GB of VRAM on Llama-3.1-8B running …
NVIDIA Corporation released an interactive 3D visualization of its H100 GPU architecture using Three.js, allowing users to explore components such as CUDA cores, HBM3 memory, GPCs, SMs, Tensor Cores, …
A Hacker News user asks what developers would do with a hypothetical CPU-friendly content generator that is orders of magnitude faster than a transformer, runs on an ordinary HP PC without H100s or 64…
Hyperscalers may face massive premiums for natural gas power as AI infrastructure demands unprecedented energy density, potentially driving up the cost per token for AI inference and forcing a pivot t…
Open-weight AI models have closed the performance gap with closed-source APIs, with specialized 7B to 30B models now outperforming older 175B models due to improved data quality and Mixture-of-Experts…
Nvidia is pursuing a $500 billion market valuation target by integrating its InfiniBand networking, CUDA software, and Blackwell architecture into a proprietary full-stack ecosystem, creating high swi…
Tax incentives, particularly accelerated depreciation on server hardware, are a major driver of the AI infrastructure boom, according to an analysis. Companies can shield significant income from taxes…
NVIDIA's data center GPU roadmap has advanced through four major architectures—Ampere, Hopper, Blackwell, and Vera Rubin—each delivering more memory, faster interconnects, and lower-precision compute …
A new analysis of 2,200 AI research reproducibility cases finds that most papers fail to replicate due to hardware variance, hyper-parameter sensitivity, and dependency issues, with the missing link o…
In a new series on LLM inference, the author explains the core concepts behind running language models in production, starting with the fundamental division between prefill and decode. The series cove…
CoreWeave has booked Nvidia A100 GPUs through 2029, nine years after the chip's May 2020 launch, CEO Mike Intrator confirmed on the company's second-quarter earnings call. Nvidia CEO Jensen Huang ampl…
CME Group, the Chicago-based exchange, announced it will launch futures contracts tied to the cost of renting Nvidia's H100 and B200 AI chips starting October 5, pending regulatory approval. The contr…
Kubernetes 1.37, releasing August 26, brings pod-level resources to stable, DRA device taints to GA, and advances GPU partitioning, addressing the 5% average GPU utilization measured by Cast AI across…
An engineer explains that the pricing difference between input and output tokens in LLM inference stems from the distinct hardware bottlenecks of prefill and decode phases. Prefill is compute-bound, w…