The serving layer is where the real cost of inference lives. For a week the industry’s attention has been on the license layer, where four labs moved to gate commercial deployment of open-weight models at the $20M revenue gate. DeepSeek just moved the pressure to the runtime. The lab that competes on price has given away the engine that runs its own production services.
We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
That is DeepSeek’s framing, in the announcement that shipped the code at 09:30Z Tuesday. The release itself is narrower than “production stack” suggests and wider than anything the four license gates hand over. Start with the terms: the Apache 2.0 license on the engine carries no additional conditions and no revenue gate, and the weights of DeepSeek-V4.1-Flash-1017 ship under MIT. Four labs now require commercial service operators to negotiate unpublished terms above $20 million in revenue. DeepSeek requires nothing of the kind.
One tree, two silicon stacks #
The DeepInfer repository carries the kernels DeepSeek runs in production, by the company’s account: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels, with confidence-scheduled decoding, bounded replay, a zero-overhead batch scheduler, and structured outputs. Hardware support runs to NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs from the same tree. The switch between them is one environment variable, DEEPINFER_ACCELERATOR. The de-Americanized serving path is not a fork. It is a build flag.
The performance language around the release is the company’s own: “high throughput and low latency” in the announcement, benchmark charts in the repository measured on 4 SM100 GPUs for prefill and 64 for decode inside a single scale-up domain. The repository publishes no numeric engine speedups. No independent benchmarks of DeepInfer exist yet. Every performance figure attached to this release is vendor-reported.
What the release withholds #
This is not a full serving stack, and the repository says so itself. The release provides offline inference. A complete inference service still requires, in the repository’s own list, a KV cache system, prefill and decode disaggregation, and load balancing across data-parallel replicas. Those three components are what separate a fast engine from a running service. The download does not include them. Whoever deploys DeepInfer in production builds them or borrows them.
The borrowing is already mapped. DeepInfer’s acknowledgments state it was “originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM.” All three are open source, and all three already carry what this release omits. vLLM ships paged attention and disaggregated prefill, decode and encode. SGLang’s ecosystem ships routing, load balancing and cluster orchestration. TensorRT-LLM ships prefill-decode disaggregation. A lab-grade engine now rests on community infrastructure the community maintains, and the integration labor lands on builders and the projects that serve them. The companion release deepseek-recipe closes the protocol gap: MIT-licensed Rust and Python components that turn API requests into DeepSeek prompts and parse the output back. Its README is plain about the boundary: “The caller must provide model inference, tool execution, and HTTP transport.” Between the two repositories a builder gets DeepSeek’s own protocol layer and engine. What remains is serving.
Where the cost lands #
Where that boundary sits is the story, because the cost of running models is the pressure point of the week. Inception’s Mercury 3 undercut speed-optimized inference pricing on Monday. doubleAI’s Maxwell, funded by Nvidia, bills customers by the compute savings it creates. DeepSeek is attacking the same layer from inside a frontier lab, framed in its announcement as lowering compute costs and energy use. When the lab that competes on price gives away its engine, the floor drops for everyone building on it. The release also lands the day the ground split further. A US federal exclusion order began reaching third-party products built on DeepSeek implementations the same Tuesday, and China’s Ministry of Commerce proposed export controls on 23 AI-stack technologies, the two moves our three-regimes analysis mapped this week: the contract layer converges while the trade layer diverges. DeepInfer serves both sides of that divide. One repository, one Apache license, and an accelerator flag that runs American and Chinese silicon from the same code. For any US-based team deploying it, the download arrives with a compliance question attached, because the exclusion order reaches products built on DeepSeek implementations. For agent workloads the engine ships the reproducibility plumbing: batch-invariant inference and deterministic sampling, both NVIDIA-only flags, plus structured outputs for constrained generation. That is the part of the serving layer agents actually need, and it is free. The rest of the service is priced in labor. The engine costs nothing. Making it a service does not, and that work has to land somewhere. Today it lands on builders.