# DeepSeek Opens DeepInfer: What Developers Still Need to Build a Serving Stack > Source: > Published: 2026-10-21 01:40:13+00:00 DeepSeek has open-sourced DeepInfer, the inference framework behind its large-scale production services, alongside deepseek-recipe, a component for converting API requests into model inputs and parsing outputs. For developers considering self-hosted DeepSeek applications, the release exposes two useful layers: model execution and protocol handling. The engineering question is how much infrastructure remains between those layers and a reliable application endpoint. DeepSeek’s [deployment Q&A](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md) makes that boundary explicit. Parts of its production system, including KV cache management and load balancing, remain tied to internal infrastructure and are not yet fully open-sourced. That distinction matters when estimating both implementation effort and potential savings. **The release gives infrastructure teams a concrete starting point.** The [DeepInfer repository](https://github.com/deepseek-ai/DeepInfer) currently documents support for DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. It provides offline inference and is licensed under Apache 2.0. Those requirements narrow the immediate audience. A team with access to supported accelerators can investigate the execution engine directly. A developer working on a laptop should begin by studying the architecture or experimenting with the separate protocol component, rather than budgeting for a local DeepInfer deployment. DeepSeek describes an architecture that separates vision encoding, prefill, and decode, allowing resources to be allocated around the demands of each stage. Its Q&A also describes distributed execution and speculative decoding. These are useful implementation references for teams already operating inference infrastructure, but they do not establish a universal cost reduction for every workload. Consider a hypothetical application that generates campaign variants overnight. Its initial requirement is a batch of completed outputs, so offline inference offers a relevant starting point. An interactive campaign assistant has additional needs: admission control, streaming delivery, cancellation, and predictable latency under concurrent traffic. Those requirements should appear in the project estimate before a team promises a production migration. **Protocol conversion is a separate integration opportunity.** The [deepseek-recipe repository](https://github.com/deepseek-ai/deepseek-recipe) contains Rust libraries with Python bindings. It converts Messages, Chat Completions, and Responses requests into a shared conversation representation, renders DeepSeek prompts, encodes tokens with an attached tokenizer, and converts generated output into the corresponding response format. The project uses the MIT license. For example, imagine a product with several clients that already submit different request formats. A shared conversion layer could reduce duplicated prompt-rendering and response-parsing code. That is an implementation opportunity, not evidence that switching an existing service will require no changes. The repository explicitly assigns model inference, tool execution, and HTTP transport to the caller. An application receiving a tool call must therefore retain its own execution policy and dispatch logic. Supporting the representation of a tool call does not supply the application that executes it. Structured output has a similar boundary. Recipe can convert format requirements and produce an output constraint, but the caller decides whether the inference backend enforces that constraint through constrained decoding. For a lead-enrichment application expecting JSON, developers should test the complete request-to-response path and validate required business fields before passing results downstream. **A first experiment can stay deliberately small.** For DeepInfer, the documented build prerequisites are Linux, GCC 13 or newer, and Python 3.12, with CUDA 13 for NVIDIA or CANN 9.2 for Ascend. The [getting-started instructions](https://github.com/deepseek-ai/DeepInfer) describe selecting the accelerator and building from source in an empty Python environment. A practical evaluation sequence is: `python examples/offline_inference.py --help` to inspect the available arguments before choosing a configuration.`--check_structured_output` when evaluating constrained generation. The documented checks cover regex, JSON, and EBNF outputs and normal end-of-sequence termination. The checkpoint handling and structured-output checks are documented in the [official examples](https://github.com/deepseek-ai/DeepInfer/blob/main/examples/README.md). These commands and steps have not been executed for this article; they describe a starting procedure, not a tested deployment recipe or a statement of hardware capacity. Keep the first workload representative. For the campaign example, include short product descriptions, longer brand guidance, and requests for several output variants. Record input and output lengths so that a faster run is not mistaken for an improvement when it simply generated less text. **The business case depends on the service around the engine.** DeepInfer’s README lists a KV cache system, vision/prefill/decode disaggregation, and load balancing across data-parallel replicas among the requirements for a complete service that are not included in the release. Although much of the engine functionality needed for disaggregation is implemented, assembling the surrounding system remains substantial work. For a growth team, the useful economic measure is cost per accepted result. Include accelerator rental, idle capacity, integration work, and operational support, then account for outputs rejected by application validation. A lower engine execution time alone does not demonstrate a lower cost for a usable campaign or completed workflow. Teams with supported hardware and inference expertise now have access to DeepSeek’s execution framework and a reusable protocol layer. The immediate opportunity is to build a bounded offline prototype, identify which surrounding components it needs, and use measured application results to decide whether operating the full service is worthwhile.