An engineer at g factor detailed how vLLM's PagedAttention and continuous iteration-level batching solve the memory-bandwidth bottleneck in production LLM inference, drawing on benchmarks run on dedic…
StepFun released Step 5 Preview, a sparse Mixture-of-Experts model with about 600B total parameters and about 27B activated per token, offering a 1M-token context window and text, image, and video inp…
RBS-Attention, a training-free sparse-prefill method from a new arXiv paper (2609.20971v1), achieves 20.65x standalone prefill-attention speedup, 11.92x vLLM prefill-attention speedup, and 5.97x end-t…
Anthropic CEO Dario Amodei is proposing to "pace" AI development through independent safety evaluators and coordination among labs in democratic countries, a plan Nvidia CEO Jensen Huang is not fully …
Researchers introduced block parallelism (BP) and context-sharded block parallelism (CSBP), new distributed parallelism dimensions for training block diffusion language models (BDLMs) on long contexts…
Engineer Rohan Bansal reported on September 16, 2026 that a 4-billion-parameter model trained on roughly 500 frontier-model example runs plus reinforcement learning against four Postgres database cont…
Agnes AI released Agnes-3.0-Flash Preview, a 33-billion-parameter open-weight multimodal language model under an Apache 2.0 license with a 262,144-token context window and a hybrid attention architect…
NVIDIA's Resiliency Extension (NVRx) can be integrated into PyTorch Fully Sharded Data Parallel training on Amazon Elastic Kubernetes Service to cut idle GPU time, with synchronous checkpointing consu…
Shopify's ML team fine-tuned a 0.8B-parameter model for buyer-profile generation that scored 84.6 against GPT-5.6-sol's 83.0 on the company's judge, according to a chart CEO Tobi Lütke posted on Septe…
Researchers from Peking University, ByteDance, and Canva developed Helios, a 14B-parameter video generation model that runs at 19.5 FPS on a single H100 GPU while cutting inference costs roughly 100x …
An independent research group in Austria has developed two quantization techniques, GSQ and RCO, that compress Z.ai's 320-billion-parameter GLM 5.3 Flash vision-language model from roughly 320GB at fu…
NVIDIA Transformer Engine is now integrated with JAX to accelerate dropless Mixture of Experts (MoE) training, targeting the conditional computation bottleneck in architectures used by DeepSeek, Qwen,…
ApexSovereign Infrastructure Protocol launched ApexSovereign, a bare-metal GPU compute service it says cuts model fine-tuning and inference overhead by up to 40% by scanning global spot-market pricing…
ApexSovereign launched a real-time GPU compute arbitrage protocol that it says scans global spot-market pricing maps every 1,500 milliseconds to secure underutilized bare-metal hardware and cut model …
Beam released Beta9, an open-source runtime for serverless AI workloads that it positions as an alternative to Modal, offering container cold starts in under a second and fan-out to hundreds of contai…
Nex-AGI released its Nex-N2.5 family of open-weight agentic models in three sizes — mini, Pro, and Max — with a prebuilt Docker image running a customized SGLang fork (nexagi/sglang:v0.5.18-nex-patch)…
Long-horizon AI agents fail at an exponential rate because a 95% per-step success probability yields only a 35.8% chance of completing 20 turns and 7.7% at 50 steps, and the chance of a second error j…
NVIDIA detailed BioNeMo Inference Runtime (BioIR), a Python library that accelerates biomolecular structure-prediction models on NVIDIA GPUs while remaining in plain PyTorch. In a matched benchmark on…
NVIDIA's NIM (NVIDIA Inference Microservices) stack delivers 2.5x higher concurrency for the Nemotron 3 Ultra model through full-stack optimizations including PagedAttention-based KV cache management,…
NVIDIA has integrated its Nemotron large language models directly into Palantir Foundry to reduce the lag between silicon fabrication and live data center deployment, targeting metrics it calls "time-…