OpenCode reported that users processed 26 trillion tokens through the anonymous AI model Ox Alpha during its first four days, reaching 327,000 unique users and 8,328,244 completed sessions, making it …
OpenAI's compute chief Sachin Katti said inference will account for over 80 percent of all AI compute spend, with a roadmap to 30GW, while Crusoe's Chase Lockmiller reported that power, not GPU supply…
A one-time snapshot of DeepSeek V4 Flash 0731 latency across nine inference providers found Baseten fastest with 3,980 decode tok/s and 7.72 s total p99, while Azure ran an older checkpoint and Scalew…
Moonshot AI released Kimi K3, a 2.8-trillion-parameter open-weight model, on July 27, 2026, making it the largest open-weight model as of August 2026. It features 104B active parameters per token, a 1…
Baseten, an AI infrastructure company that raised a $1.5B Series F led by Altimeter Capital, Conviction Partners, and Spark Capital, is hiring a Software Engineer for AI Developer Productivity in San …
CodeRabbit, NVIDIA, and Baseten post-trained NVIDIA Nemotron 3.5 Lightning to route code reviews, achieving 80.7% exact route agreement (up from 75.8% for the GPT baseline) and cutting estimated infer…
Nvidia Corp. unveiled Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts open model, and NeMo Switchyard, an open-source routing library, to deliver faster and more efficient agentic AI…
Moonshot AI released the weights for its Kimi K3 model with 2.8 trillion parameters and a 1M token context window under a new Kimi K3 License, which allows inference providers like Modal, Baseten, Fir…
Hugging Face has added Baseten as a supported Inference Provider on its Hub, enabling serverless access to open-weight LLMs such as Kimi K3, DeepSeek V4 Flash, and GLM-5.2 for conversational and text-…
Moonshot AI released the weights for Kimi K3 on July 27, a 2.8-trillion-parameter Mixture-of-Experts model with a 1-million-token context window, scoring third globally behind Claude Fable 5 Max and G…
Baseten, an AI inference platform, raised $300 million in a Series E round at a $5 billion post-money valuation, co-led by Institutional Venture Partners (IVP) and CapitalG, Google's growth fund, with…
IBM's article explains that static, dynamic, and continuous batching are methods to improve GPU utilization in large language model inference, with continuous batching scheduling at the token level to…
Three independent benchmarks show that expensive AI defaults are rarely optimal: medium reasoning effort captures nearly all of Claude Opus 5's bug-fix gains, a three-model open-weight jury (GPT-OSS 1…
Baseten, an AI infrastructure company, has raised a $13 billion Series F round, becoming one of the new AI infrastructure decacorns. In a recent episode of the Latent Space podcast, Baseten's Philip K…
Baseten, an AI infrastructure company that recently raised a $1.5B Series F led by Altimeter Capital, Conviction Partners, and Spark Capital, is hiring an AI Inference Engineer in San Francisco with a…
NVIDIA Dynamo, an Apache-2.0 datacenter-scale inference orchestration layer from NVIDIA, sits above vLLM, SGLang, and TensorRT-LLM to coordinate multi-GPU clusters, with published benchmarks showing u…
Thinking Machines' Inkling, a 975B total / 41B active MoE model with Apache 2.0 license and 1M context, is being trialed via a bash wrapper alias 'claude-inkling' that bridges Anthropic's CLI to the O…
The latest version of the Model Context Protocol (MCP) introduces a stateless protocol core, shifting from a bidirectional stateful protocol to a request/response model, according to lead maintainers …
A Baseten inference engineer known as @waterloo_intern published a technical blog post titled '22,580: From GPT-2 to Kimi K3, Explained,' which has garnered 2.4 million views. The post provides runnab…
The marginal cost of AI inference will approach zero as open models rival frontier models, reshaping the business models of labs like OpenAI and Anthropic toward low-margin inference sales, according …