An engineer's analysis reveals that LLM inference throughput on NVIDIA GPUs is bottlenecked by memory bandwidth, not compute. A 7B model achieves ~15 tok/s on a T4 but over 3,500 tok/s on an H100, a 1…
Nvidia has engineered a circular financing loop with GPU cloud providers CoreWeave and Nebius, selling them GPUs, investing billions, and guaranteeing to buy unsold capacity, creating a $35 billion de…
Researchers introduced two CUTLASS-based kernels that accelerate the SwiGLU activation function in large language models by up to 2.47x on NVIDIA H100 GPUs, shifting workloads from memory-bound to com…
Bryan Oliver, a Thoughtworks Radar contributor and author, presented on chaos engineering for GPU clusters at a recent event, highlighting the complexity and scale of modern AI hardware such as NVIDIA…
NVIDIA announced the BioNeMo Agent Toolkit to accelerate biomolecular structure prediction and co-folding with OpenFold3, using GPU-accelerated MSA generation and multi-GPU scaling to enable virtual s…
Researchers propose Jet-Long, a tuning-free zero-shot method for extending LLM context windows that dynamically adjusts a rescaling factor based on sequence length, preserving short-context fidelity w…
NVIDIA AI released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed hybrid MoE LLM that achieves up to 2.14x server throughput on 8xB200 nodes and enables 8 concurrent 1M-token requests on a single H100, …
DeepSeek released V4 Flash, a 284B-parameter MoE model with 13B active parameters per token, featuring FP4+FP8 hybrid attention and 1M native context. The open-weight model, available under MIT licens…
NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed hybrid MoE LLM that achieves 2.03x server throughput over Nemotron-3-Super at matched user throughput. The model uses iterative puzzle comp…
Hugging Face announced that its Transformers modeling backend for vLLM now matches or exceeds native vLLM speed for compatible architectures, reducing the need for separate hand-optimized serving port…
A new model estimates that a 10x reduction in R&D compute for an AGI company would slow AI progress by about 6x in the median case, with an 80% confidence interval of 3.5x to 8x. The model accounts fo…
A PhD researcher in AI and automated theorem proving is seeking access to a multi-GPU server capable of running DeepSeek-Prover-V2-671B, offering to pay for GPU usage and interested in long-term colla…
Chinese firms are shifting from Nvidia to domestic AI chips as Beijing certifies nine homegrown processors for state procurement, potentially collapsing Nvidia's China market share from 66% to 8% by 2…
Cerebras is running OpenAI's GPT-5.6 Sol at 750 tokens per second on its WSE-3 chip, a 10x throughput improvement over Nvidia H100 clusters. This drastically reduces agentic workflow latency, but acce…
Cast AI's 2026 analysis of tens of thousands of production clusters on AWS, GCP, and Azure found average CPU utilization at 8% fleet-wide, down from 10% the prior year. The company identified seven ke…
Austria's MUSICA supercomputer, featuring 1,088 NVIDIA H100 GPUs delivering 45.11 petaflops, was inaugurated on July 3, 2026, marking an eightfold increase over previous systems. The EUR 45 million sy…
IEEE Spectrum reports that orbital data centers remain economically and technically impractical despite Elon Musk's predictions, citing massive scaling challenges in launch capacity, satellite manufac…
SpaceX founder Elon Musk claimed at Davos that orbital data centers will be the lowest-cost AI option within two to three years, and SpaceX filed an FCC application for up to 1 million satellites. How…
Speculative decoding accelerates large language model inference by 20-50% without quality loss, using a draft-verify mechanism that generates multiple tokens per forward pass. The technique amortizes …
LoRA (Low-Rank Adaptation) and QLoRA have become widely adopted methods for efficiently fine-tuning large language models with a fraction of the parameters, solving the problem of massive GPU requirem…