Modal, Fireworks, and Baseten gain cost advantage with Nvidia and AMD chips for Kimi K3

Modal, Fireworks AI, and Baseten are offering hosted inference for Moonshot AI's Kimi K3 at roughly one-tenth the cost of direct access, using Nvidia GB300 NVL72 systems and AMD MI350X and MI355X accelerators that US export controls largely keep from Chinese firms. Moonshot AI released Kimi K3 on July 27, 2026, a 2.8-trillion-parameter Mixture-of-Experts open-weights model that routes each token through 16 of 896 experts, activating about 104 billion parameters per forward pass, with a 1-million-token context window and weight files exceeding 1.4 terabytes under MXFP4 quantization. Modal reported 460 tokens per second on its Kimi K3 deployments using a custom speculative decoder called DFlash, with shared endpoints priced at approximately $3 per million input tokens, $0.30 per million cached tokens, and $15 per million output tokens.

Photo: Matheus Bertelli / Pexels Modal, Fireworks, and Baseten gain cost advantage with Nvidia and AMD chips for Kimi K3 US inference providers can serve Moonshot AI's massive open-weights model at a fraction of the cost thanks to superior GPU hardware Three American AI infrastructure companies have found a lucrative edge in an unlikely place: serving a Chinese-built model better and cheaper than its creators can. Modal, Fireworks AI, and Baseten are now offering hosted inference for Moonshot AI’s Kimi K3 at roughly one-tenth the cost of direct access, powered by Nvidia https://cryptobriefing.com/markets/nvidia/ ’s and AMD https://cryptobriefing.com/markets/amd/ ’s latest accelerator hardware that remains largely unavailable to Chinese firms under US export controls. What makes Kimi K3 worth the effort Kimi K3 is not a small model. Released by Moonshot AI on July 27, 2026, it packs 2.8 trillion parameters into a Mixture-of-Experts architecture, making it one of the largest open-weights models ever published. The “open weights” distinction matters: anyone can download and run the model, which is exactly what Modal, Fireworks, and Baseten have done. The MoE design means the full 2.8 trillion parameters don’t activate on every query. Instead, K3 routes each token through 16 of its 896 available experts, activating roughly 104 billion parameters per forward pass. K3 also supports a context window of 1 million tokens, which means it can ingest and reason over book-length documents in a single session. But running a model this size is not trivial. The weight files exceed 1.4 terabytes even when compressed using MXFP4 quantization. Baseten has noted that deployments require multi-GPU configurations, with setups like eight Nvidia GB300 accelerators working in concert just to hold the model in memory. The hardware advantage US inference providers have access to Nvidia’s GB300 NVL72 systems and AMD’s MI350X and MI355X accelerators. Chinese companies, including Moonshot AI itself, face export restrictions that limit their access to these exact processors. AI, tech, and the markets they move—in one daily briefing. Daily. Free. Join 34,000+ readers across crypto, finance, and policy. Modal reported that its deployments of Kimi K3 achieve 460 tokens per second using a custom speculative decoder called DFlash, purpose-built for K3’s architecture. The pricing reflects these efficiencies. Shared endpoints across the three providers land at approximately $3 per million input tokens, $0.30 per million cached tokens, and $15 per million output tokens. Compliance as a selling point Beyond raw performance, Modal, Fireworks, and Baseten are marketing something arguably more valuable to their target customers: data residency and compliance guarantees. All three emphasize zero-data retention policies, meaning customer prompts and completions are not stored or used for training. They also offer dedicated capacity and US-hosted endpoints, features that matter enormously to enterprises navigating data sovereignty regulations in sectors like healthcare, finance, and government contracting. The three providers also support OpenAI-compatible APIs, which lowers the switching cost for enterprises already integrated with OpenAI’s tooling. A developer can point their existing code at a Kimi K3 endpoint with minimal changes, testing whether the 2.8 trillion-parameter model outperforms their current setup without a major engineering investment. Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy https://cryptobriefing.com/editorial-policy/ .