DeepSeek gave away its engine. The $38-an-hour parts stayed home

DeepSeek released DeepInfer, the inference engine behind its own API, as open source, but the release covers only offline inference: the README lists prefill and decode load balancers and KV cache storage as "not included in this release," and a deployment Q&A states those components are tightly coupled to internal infrastructure. An independent analysis found only one of five traffic-routing and cache components ships in the repo, and DeepSeek's own Q&A puts revenue at $38 per GPU-hour for its hosted service.

Originally published at tensorust.com https://tensorust.com/articles/deepseek-deepinfer/ . TL;DR: At 5:30 AM ET today DeepSeek called DeepInfer, the engine behind its own API, "now fully open source". 1.5 million people saw that post in five hours. DeepSeek's own README lists three things a real service needs that are "not included in this release", and its own Q&A says each GPU in that service earns $38 an hour . I cloned the repo and measured it this morning: 4 of the 5 boxes in DeepSeek's own diagram that route traffic or keep the cache are missing, only 6.82% of the code matches vLLM, and the Huawei kernels are 52% bigger than the Nvidia ones. So no, @zephyr z9, they did not "literally release the secret sauce behind their economics". They released the oven. The recipe that turns GPUs into $38 an hour is still in Hangzhou. It's still the best gift any AI lab has given developers this year, and it's still not what the post says it is. DeepInfer doc/inference deployment qa.md and computeprices.com's Google Cloud B200 page, both read Oct 20 2026. Revenue is not profit. Static frame: hero.png. Here is the post, from @deepseek ai at 5:30 AM ET https://x.com/deepseek ai/status/2112476496283775135 . By 10:31 AM it had 1.5M views and 19K likes. The repo hit 8,663 stars https://github.com/deepseek-ai/DeepInfer by 10:33 AM, and the Hacker News thread was 1 with 1,222 points and 437 comments https://news.ycombinator.com/item?id=50163277 . @deepseek ai on X, Oct 20 2026, captured 10:31 AM ET Oct 20 2026, real and unedited. Now open the README. Below "Supported hardware" there is a list with this heading, word for word: "Required for a complete inference service not included in this release " : The first line of the README is just as honest: "This open-source release provides offline inference ." There's no API server in the repo. I grepped it: the only HTTP code is a profiling endpoint you switch on with an environment variable. You can run prompts through DeepInfer on your own cluster. You can't point a customer at it. Plain English for the jargon. Prefill is the model reading your prompt. Decode is the model writing the answer, one token at a time. KV cache is the model's notes on a prompt it has already read, so it doesn't have to read it again. Disaggregation means prefill and decode run on separate machines. A load balancer decides which machine gets your request. Every one of those is what turns an engine into a business. DeepSeek published a deployment Q&A https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference deployment qa.md with the release. It has an architecture diagram of the whole V4.1 Flash serving system. I labelled each box from the README's "not included" list and a read of the code. Diagram from DeepInfer doc/images/architecture overview.png Apache-2.0 ; labels are ours, from the README's not-included list and a read of the code, Oct 20 2026. The Q&A names five pieces that route traffic or keep the cache: a Prefill Load Balancer, a Decode Load Balancer, a Vision Encoder Load Balancer, an Expert-Parallel Load Balancer, and KVCache Storage . Only the expert-parallel one ships it's in deepinfer/model executor/eplb/ . That's 1 of 5 . The 3FS file system under the cache has been open since February 2025 https://github.com/deepseek-ai/3FS . The component that runs the cache on top of it, which the Q&A calls kvcache-store , is not in the repo. And DeepSeek tells you why, in the Q&A, in a paragraph most people scrolled past: "The KV cache management and load balancing components of the inference system are tightly coupled to our internal infrastructure, so that code is not yet fully open-sourced." "Not yet fully open-sourced" and "now fully open source" are both DeepSeek sentences, published the same morning. Only one of them is in the tweet. This is the part that makes the "they gave away their economics" takes wrong. DeepSeek listed three design goals for the system. Goal three: "Retain KV cache for more than 72 hours" , so a repeated prompt skips the expensive reading step. That is where cheap inference comes from: you don't pay to read the same 50-page codebase twice. @zephyr z9 and @Polymarket on X, Oct 20 2026, captured 10:31 AM ET Oct 20 2026, real and unedited. Look at what the Q&A says those missing parts are worth: Now the comparison nobody in the replies made. Google Cloud rents a B200, the same Blackwell class as DeepInfer's "SM100" target, for $4.95 an hour on spot, $7.09 on a 3-year deal, $11.12 on a 1-year deal https://computeprices.com/providers/google/gpus/b200 prices updated today . DeepSeek's number is 3.4x to 7.7x what the GPU costs to rent. Revenue isn't profit, and DeepSeek owns hardware instead of renting it, but that spread is the economics. And the spread lives in the 4 boxes that aren't in the repo. Hot take, and I'll stand on it: @Polymarket's "dramatically reduce the computing costs of running AI models" is true for maybe 50 companies on Earth. Here's why. The Q&A is blunt about the minimum: "the smallest efficient deployment unit is 64 GPUs of prefill plus 64 GPUs of decode." The README benchmarks were "measured on 4 SM100 GPUs for prefill and 64 for decode". Decode wants EP64 or more the experts of the model spread across 64 to 128 GPUs . So I did the bill. Our math: 128 x $4.95 and 128 x $11.12 Google Cloud B200, Oct 20 2026 , 128 x 14.4 TB. GPU count and storage ratio from DeepInfer's Q&A. Compare that with the other DeepSeek thing on the timeline this week: Unsloth's 1-bit V4.1 Flash https://x.com/UnslothAI/status/2112212605930778830 , 508GB shrunk to 250GB, running on a 128GB+ RAM machine. That's the open model most people will actually touch. DeepInfer is for clouds. If you're a solo dev, DeepSeek's real gift to you is still the cheap API https://openrouter.ai/deepseek , not this repo. Their Q&A even has the best line of the whole release. Can you split prefill and decode across data centers? Yes, "at some cost in reliability, e.g., the fiber running between data centers is easily severed by construction work." Respect. That's a team that has had a backhoe ruin its week. The other big reaction came from the vLLM team, the open-source engine most of the industry runs on. Their post is gracious, and DeepSeek's README does say DeepInfer "was originally built primarily on vLLM". @vllm project on X, Oct 20 2026, captured 10:31 AM ET Oct 20 2026, real and unedited. So I measured how much vLLM is left. I cloned DeepInfer commit 774ce24 and vLLM at nine releases, from v0.2.0 2023 to today's main branch. I took every Python line of 25+ characters in DeepInfer's deepinfer/ package, dropped comments and blank lines, collapsed whitespace, and checked each line for an exact match anywhere in each vLLM release. The script is 60 lines of Python, ran in about a minute, and needed no GPU. python def norm l : l = l.strip if not l or l.startswith ' ' : return None if len l < 25: return None skip trivial lines return re.sub r'\s+', ' ', l every .py file in each vLLM tag, read straight out of git p = subprocess.run 'git', '-C', VL, 'cat-file', '--batch' , input='\n'.join f'{ref}:{f}' for f in files .encode , capture output=True Our run, 10:20 AM ET Oct 20 2026: DeepInfer 774ce24 vs vllm-project/vllm tags and main 255e3a3, exact match of whitespace-normalized lines of 25+ characters. Result: 1,711 of 25,086 lines 6.82% appear in at least one vLLM release. The best single release is today's vLLM main at 5.70%, which mostly means both projects picked up the same new ideas. The most vLLM-flavored file is shm broadcast.py , a shared-memory message queue, at 76.7%. The engine core, llm engine.py , is 19.2%. The V4 model code is around 5 to 12%. The folder layout still says vLLM: worker/cache engine.py , sequence.py , model runner.py . Those are the names vLLM used back in 2023. But the code inside got rewritten. This method misses renamed variables and refactors, so treat 6.82% as the floor of the family resemblance, not a plagiarism score. My read: "built on vLLM" is like saying a skyscraper is built on the parking lot that used to be there. True, and generous, and DeepSeek gets real credit for writing the other 93%. Here's what I think the release is really about. DeepInfer's custom kernels the small GPU programs that do the math live in tilelang ops/ , with one backend per chip. Same 30 kernels, two versions each. I counted non-blank lines. Line counts from DeepInfer 774ce24, 10:25 AM ET Oct 20 2026; repo dates from the GitHub API; ban per Reuters, Oct 19 2026. Some of that +52% is just the cost of a younger chip: Ascend kernels need more hand-holding than CUDA ones. But nobody writes 6,793 lines for a chip they don't plan to sell on. Here's what I think is really happening, and they'll never say it out loud: this is a Huawei sales deck with an Apache license. Every Chinese cloud that wants to serve DeepSeek at DeepSeek's prices now has a production-grade engine that runs on chips the US can't cut off. The US bans DeepSeek from its agencies on Monday. On Tuesday DeepSeek makes its model cheaper for everyone else to serve, on Huawei silicon. Lowkey the best-timed open-source drop of the year. One more receipt for the "it's a Chinese production system, not a demo" pile: 1,147 of 40,638 non-blank lines in the package carry Chinese comments or docstrings, like """compress 内核固定开启 fast math。""" "the compress kernel always turns on fast math" . Nobody wrote this for a launch blog. The whole history is 3 commits : one "initial commit" of 345 files and 63,326 lines at 12:09 AM ET, then two doc updates. It got exported from somewhere private this morning. To be fair to DeepSeek, and unfair to everyone else: OpenAI's open release was gpt-oss in August 2025 https://openai.com/index/introducing-gpt-oss/ , weights under Apache 2.0 with reference code. OpenAI, Anthropic and Google have published zero lines of the systems that actually serve ChatGPT, Claude or Gemini to paying users. DeepSeek just published the engine, the kernels, the expert balancer, a tokenizer service written mostly in Rust deepseek-recipe https://github.com/deepseek-ai/deepseek-recipe , its storage layout, its revenue per GPU-hour and its traffic. So pick your villain. Is it the lab that ships 93% of a production engine and calls it 100%, or the labs that ship 0% and call themselves the good guys? Prediction: by Nov 30, vLLM and SGLang ship most of DeepInfer's tricks the draft-token decoding, the bounded SWA replay and US clouds serve V4.1 Flash faster than they do today. And by Dec 31, nobody outside China runs it the way DeepSeek does, with the 72-hour cache and the four balancers, because DeepSeek kept those. If I'm wrong, show me the cluster and I'll write the correction with your name in the headline. Your turn: is "fully open source" fair for a release you can't serve a single API request with? Yes or no, and say why in the replies. @deepseek ai, you're welcome to answer too. Want to see what V4.1 Flash costs you through an API versus renting the GPUs yourself? The LLM API Cost Calculator $7, Google Sheets + Excel, 26 models https://shop.tensorust.com/buy/llm-cost-calculator?utm source=devto&utm campaign=deepseek-deepinfer does the per-request math in one sheet. The free version is on the site at tensorust.com/tools/llm-api-cost/ https://tensorust.com/tools/llm-api-cost/ . Related on tensorust.com: Meta open-sourced an 816B model. Running it takes about 1TB of RAM https://tensorust.com/articles/muse-spark-open-weights-ram/ , LLM proxy benchmark: 200 duplicate calls became 1 https://tensorust.com/articles/llm-proxy-benchmark/ , and Theo's 60% stat is an outage chart https://tensorust.com/articles/claude-code-vs-codex-outages/ . Method and data: DeepInfer cloned at 774ce24 Oct 20 2026 , vllm-project/vllm tags v0.2.0, v0.2.7, v0.3.3, v0.4.3, v0.5.5, v0.6.6, v0.8.5, v0.10.2 and main at 255e3a3. All counts run 10:20 to 10:35 AM ET Oct 20 2026. X screenshots captured 10:31 AM ET, real and unedited. GPU prices from computeprices.com's Google Cloud B200 page, updated Oct 20 2026.