Inception released Mercury 2.5, which it calls the most capable diffusion LLM on the market and the largest it has ever trained, reporting a 40% intelligence gain over Mercury 2 at 1,107 tokens per se…
OpenRouter retired behavioral monitoring of the z-ai/glm-5.3-flash endpoint hosted at baseten/fp8 on 2026-09-17 after re-initialization repeatedly timed out following a detected change, leaving no log…
Inception AI released Mercury 2.5, its most capable production model, claiming a 40% increase in intelligence over Mercury 2 with speeds of 1,107 tokens per second and a context window of 260K tokens,…
Harvey, an AI company for legal and professional services, is hiring a Staff Software Engineer for Model Infrastructure in San Francisco with a salary of $231k–340k/yr. The role involves leading the d…
Baseten's technical breakdown of LLM inference efficiency identifies two categories of engineering choices: those that trade latency for throughput, such as batch sizing, tensor parallelism, expert pa…
Baseten, the AI inference company valued at $13 billion, published a technical framework on September 1st that distinguishes genuine efficiency gains in large language model inference from tradeoffs t…
Z.ai, the Chinese company formerly known as Zhipu AI, revealed on August 26, 2026, that its anonymous Ox Alpha model was GLM-5.3-Flash, an open-weight 320-billion-parameter mixture-of-experts model pr…
AI startup Yutori, co-founded by former Meta AI researchers Devi Parikh, Dhruv Batra, and Abhishek Das, announced n2, a 27-billion-parameter model designed for cost-effective computer use, claiming pe…
OpenCode reported that users processed 26 trillion tokens through the anonymous AI model Ox Alpha during its first four days, reaching 327,000 unique users and 8,328,244 completed sessions, making it …
OpenAI's compute chief Sachin Katti said inference will account for over 80 percent of all AI compute spend, with a roadmap to 30GW, while Crusoe's Chase Lockmiller reported that power, not GPU supply…
A one-time snapshot of DeepSeek V4 Flash 0731 latency across nine inference providers found Baseten fastest with 3,980 decode tok/s and 7.72 s total p99, while Azure ran an older checkpoint and Scalew…
Moonshot AI released Kimi K3, a 2.8-trillion-parameter open-weight model, on July 27, 2026, making it the largest open-weight model as of August 2026. It features 104B active parameters per token, a 1…
Baseten, an AI infrastructure company that raised a $1.5B Series F led by Altimeter Capital, Conviction Partners, and Spark Capital, is hiring a Software Engineer for AI Developer Productivity in San …
CodeRabbit, NVIDIA, and Baseten post-trained NVIDIA Nemotron 3.5 Lightning to route code reviews, achieving 80.7% exact route agreement (up from 75.8% for the GPT baseline) and cutting estimated infer…
Nvidia Corp. unveiled Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts open model, and NeMo Switchyard, an open-source routing library, to deliver faster and more efficient agentic AI…
Moonshot AI released the weights for its Kimi K3 model with 2.8 trillion parameters and a 1M token context window under a new Kimi K3 License, which allows inference providers like Modal, Baseten, Fir…
Hugging Face has added Baseten as a supported Inference Provider on its Hub, enabling serverless access to open-weight LLMs such as Kimi K3, DeepSeek V4 Flash, and GLM-5.2 for conversational and text-…
Moonshot AI released the weights for Kimi K3 on July 27, a 2.8-trillion-parameter Mixture-of-Experts model with a 1-million-token context window, scoring third globally behind Claude Fable 5 Max and G…
Baseten, an AI inference platform, raised $300 million in a Series E round at a $5 billion post-money valuation, co-led by Institutional Venture Partners (IVP) and CapitalG, Google's growth fund, with…
IBM's article explains that static, dynamic, and continuous batching are methods to improve GPU utilization in large language model inference, with continuous batching scheduling at the token level to…