DeepSeek gives away the engine behind its cheap AI, 18 months after saying it wouldn’t

DeepSeek published DeepInfer, the inference framework that powers its production services, on GitHub on Tuesday under the Apache 2.0 licence, 18 months after the Chinese lab said it would not release its full inference engine. DeepInfer runs on Nvidia Blackwell chips and Huawei Ascend 950, supports only the DeepSeek-V4.1-Flash model (October 17 checkpoint), and ships DeepSeek's own kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect), DSpark speculative decoding, and structured outputs, but omits a KV cache system, prefill/decode machine splitting and cross-replica load balancing. DeepSeek's benchmark charts were measured on 4 GPUs for prefill and 64 for decode, and the framework requires Linux, Python 3.12 and either CUDA 13 or Huawei's CANN 9.2.

An Asus server rack built on Nvidia’s GB200 NVL72 Blackwell system at Computex 2024 in Taipei illustrative . Image: 极客湾 Geekerwan <https://commons.wikimedia.org/wiki/File:Eindr%C3%BCcke von der COMPUTEX 2024 %E6%9E%81%E5%AE%A2%E6%B9%BEGeekerwan 19.png / Wikimedia Commons, CC BY 3.0 https://creativecommons.org/licenses/by/3.0/ , cropped DeepSeek has released the software that runs its own AI service, 18 months after saying it wouldn’t. The Chinese lab published DeepInfer on GitHub https://github.com/deepseek-ai/DeepInfer on Tuesday, describing it as “the high-performance inference framework that powers DeepSeek’s large-scale production services”, and it runs on Nvidia’s Blackwell chips and Huawei’s Ascend 950 alike. Inference is the part of AI that happens after training: taking a finished model and serving answers to millions of people quickly and cheaply. DeepSeek’s low prices have always rested on how well it does this, and DeepInfer is the first time it has shown the whole engine rather than individual parts. The code is under the Apache 2.0 licence, so anyone can use, change and sell it. That is a turnaround. In a note in April 2025 https://github.com/deepseek-ai/open-infra-index/tree/main/OpenSourcing DeepSeek Inference Engine , DeepSeek said it had considered releasing its full inference engine but decided against it, because the code was a heavily customised early fork of vLLM, was “tightly coupled” with its internal cluster tools, and its small team lacked the bandwidth to maintain a big open-source project. It promised to contribute pieces to existing projects instead. DeepInfer’s README still credits vLLM as the base it was “originally built primarily on”, with ideas from SGLang and Nvidia’s TensorRT-LLM. 🚀 Introducing DeepInfer, our in-house inference framework, now fully open source Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepInfer https://github.com/deepseek-ai/DeepInfer 1/4 October 20, 2026 https://x.com/deepseek ai/status/2112476496283775135 What developers get For now DeepInfer supports one model, DeepSeek-V4.1-Flash https://madrobot.blog/2026/10/17/deepseek-v4-1-flash-1017-beats-claude-opus-5-benchmarks-price/ , the cheap model DeepSeek upgraded on Saturday, and it downloads the October 17 checkpoint from Hugging Face on first run. The features listed in the README are the tricks DeepSeek uses in production: - DeepSeek’s own kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, plus custom TileLang code. - Splitting the work across chips: data and context parallelism for attention, and expert parallelism with load balancing for the model’s mixture-of-experts layers. - Faster decoding: a system called DSpark drafts several tokens ahead and uses confidence estimates to decide when to check them. - Repeatable answers: results that stay identical whatever else is in the batch, and deterministic sampling with a fixed seed, both on Nvidia only. - Structured outputs: answers forced to match a regex, JSON schema or grammar. DeepSeek’s benchmark charts were measured on 4 GPUs for prefill and 64 for decode. It needs Linux, Python 3.12 and either CUDA 13 or Huawei’s CANN 9.2. What isn’t included This is an offline inference release, and DeepSeek is upfront that it is not a full service. The README lists three things a complete deployment needs that are missing: a KV cache system, which stores earlier parts of conversations so they don’t have to be recomputed; splitting prefill and decode onto separate machines, though it says “most required engine functionality is already implemented”; and load balancing across replicas. Those are the pieces that turn a fast engine into a cheap public API, so companies hoping to copy DeepSeek’s pricing will still have to build them or borrow them from vLLM or SGLang. Nvidia and Huawei, side by side The hardware list has two entries: Nvidia’s SM100 GPUs, the Blackwell data-centre generation, and Huawei’s Ascend 950 NPUs. It follows DeepSeek’s release last month of Ascend versions of its core libraries https://madrobot.blog/2026/09/30/deepseek-huawei-ascend-open-source-tilelang-nvidia-cuda-alternative/ , and its founder’s push to train its biggest models on Chinese chips https://madrobot.blog/2026/09/21/deepseek-huawei-chips-8-trillion-model-must-succeed/ . A production-grade engine that treats the Ascend as a first-class target is the kind of software Huawei has lacked against Nvidia’s CUDA. It also lands the day after the US moved to ban DeepSeek from every federal civilian agency https://madrobot.blog/2026/10/19/us-bans-deepseek-federal-agencies-dhs-order-remove-third-party/ , an order that covers “third-party products and services developed using DeepSeek implementations”. The order does not name products or say whether open code counts, so any US vendor building on DeepInfer and selling to government will want an answer first. Our guide to where DeepSeek is banned https://madrobot.blog/2026/10/20/is-deepseek-banned-countries-us-uk-india-australia-canada/ covers the rest of the world. Why it matters DeepSeek has handed out the recipe behind its rock-bottom prices, minus the parts that make it a service. That lowers the bar for anyone serving its models cheaply on Nvidia or Huawei hardware, even as Washington tries to keep DeepSeek out of government. Sources: DeepSeek, DeepInfer on GitHub https://github.com/deepseek-ai/DeepInfer ; DeepSeek on X https://x.com/deepseek ai/status/2112476496283775135 ; DeepSeek, “The Path to Open-Sourcing the DeepSeek Inference Engine” https://github.com/deepseek-ai/open-infra-index/tree/main/OpenSourcing DeepSeek Inference Engine .