Baseten engineers Philip Kiely and Ali Taha detailed the shift to disaggregated prefill/decode serving, predictive layer-wise quantization error cancellation, hardware interconnect bottlenecks, and st…
Serving an open-weight large language model in a user-facing product is primarily a latency problem, and open weights enable engineers to control inference speed through levers such as weight quantiza…
Baseten, the AI inference company valued at $13 billion, published a technical framework on September 1st that distinguishes genuine efficiency gains in large language model inference from tradeoffs t…
Philip Kiely's new book, 'Inference Engineering,' is now available as a digital download, offering a comprehensive guide to the technologies and techniques powering AI inference across runtime, infras…
Baseten, an AI infrastructure company, has raised a $13 billion Series F round, becoming one of the new AI infrastructure decacorns. In a recent episode of the Latent Space podcast, Baseten's Philip K…