DeepSeek open-sourced DeepInfer to cut AI inference costs. The numbers it revealed instead expose how tight GPU supply has become for everyone, DeepSeek included.
On October 20, DeepSeek open-sourced DeepInfer, the inference engine it uses to run V4.1-Flash in production, and the numbers it disclosed are the real story. A single cluster running the framework processes 155 trillion tokens a day and generates roughly $38 an hour per GPU at off-peak pricing, according to data highlighted by Latent.Space's AI News roundup. A third-party researcher cited in that analysis put hardware-level gross margins on that revenue at somewhere between 84% and 96%, before accounting for R&D, bandwidth, or operations. That is an extraordinary return per chip. And it is exactly why demand for the chips themselves isn't easing.
V4.1-Flash itself came out September 10, a 552-billion-parameter mixture-of-experts model built on what DeepSeek calls a causal encoder-decoder design. Only about 8 billion parameters activate per token on input and 16 billion on output, and DeepSeek says the architecture cuts key-value cache requirements to roughly a quarter of its predecessor's memory footprint. The practical result: a typical request with 100,000 input tokens and 20,000 output tokens costs around five cents to run, at DeepSeek's own posted price of $0.30 per million input tokens and $1.20 per million output tokens, according to pricing data from LLM-Stats. The model is now the top-ranked open-weight system on the Vals Index, and on the DeepSWE v1.1 coding benchmark it scored 74.2, narrowly ahead of Anthropic's Claude Opus 5 at 74.0 and OpenAI's GPT-5.6 Sol at 73.0.
Cheap, fast inference pulls in more usage than it saves in compute, straining the same finite pool of Nvidia chips.
H100 rental rates have climbed close to 40% since last year, from about $1.70 to roughly $2.35 per GPU-hour, according to SemiAnalysis's newly launched H100 rental price index, which surveyed more than 100 industry participants. SemiAnalysis attributes the jump to capacity that is effectively sold out across neoclouds, the specialized GPU rental firms that have taken on heavy debt to buy chips ahead of demand. That financing structure is precisely why investors have grown nervous. Neoclouds borrowed against future utilization, and a cluster sitting idle is a cluster that can't service its debt.
Huawei and Cambricon Are Charging More for AI Chips as Memory Runs Short Huawei and Cambricon have both hiked prices sharply on unreleased AI chips as a high-bandwidth memory shortage, worsened by US export controls since December 2024, pushes them onto costly grey-market supply channels. Huawei's Ascend 950DT now runs over $37,000 a card, even as the company races to keep up with domestic demand it says already... - Huawei Ascend 950DT AI chip price increase 2026 - memory shortage driving premium prices for accelerators
V4.1-Flash doesn't change that math by competing for the same scarce chips in a vacuum. It changes it by making inference cheap enough that far more of it gets run. DeepSeek's own model is now live across 21 third-party inference providers, including Fireworks AI, Together AI, DeepInfra, Baseten, Novita, and Parasail, according to pricing comparisons tracked by Artificial Analysis. Fireworks prices it at $0.24 per million tokens, Together AI at $0.32, and DeepInfra at $0.61, each undercutting DeepSeek's own $0.26 rate in places and racing each other on throughput. That's a lot of competing infrastructure chasing the same workload. Every one of those providers needs GPU-hours to serve it.
DeepInfer complicates the picture further because DeepSeek built it to run on more than just Nvidia silicon. The framework, released under Apache 2.0, supports Nvidia's Blackwell-generation chips alongside Huawei's Ascend 950, according to reporting from TechNode. That's a deliberate hedge against the export restrictions that have limited DeepSeek's access to top-tier Nvidia hardware, and it signals DeepSeek expects the GPU squeeze to outlast any one generation of chips.
Open weights, compute-maximalist rivals #
The contrast with Western frontier labs is getting sharper. OpenAI and Anthropic have largely competed by throwing more compute at bigger models and signing multi-billion-dollar capacity deals years in advance. DeepSeek is doing the opposite: publishing the weights, publishing the serving code, and daring anyone with a GPU cluster to undercut its own prices. Both strategies end up pointed at the same bottleneck. A compute-maximalist lab needs more chips to train the next model. An efficiency-maximalist lab needs more chips because its model got cheap enough that everyone wants to run it constantly. Frankly, that's the uncomfortable truth hiding under all the talk of "efficiency gains" in AI this year: cheaper inference has never once led to less total GPU demand. It leads to more.
Also read: Cerebras bets on wafer-scale chips as Nvidia's GPUs get scarce and pricey • Gamers are turning old Nvidia GPUs into cheap AI chips as rental prices spike • H100 rental prices are spiking even as Wall Street worries about an AI bubble
This article is posted in AI News, check it out for more related stories.
Join the discussion #
Open in the community → Almost there. Sign in and your reply posts straight away.