How to Deploy DeepSeek-R1 & Qwen on Azure with Scale-to-Zero ($0 Idle Bill)

A developer built an open-source FastAPI gateway that serves DeepSeek-R1 and Qwen models through an OpenAI-compatible API on Azure Container Apps, configured with min_replicas set to 0 so idle deployments scale to zero and incur no compute charges. The gateway exposes /v1/chat/completions and /v1/models with Server-Sent Events streaming via TextIteratorStreamer, letting existing LangChain, LlamaIndex or OpenAI SDK clients switch over by changing only their base_url. The code is published on GitHub as azure-model-deployer, with a paid boilerplate offering CUDA 12.1 GPU Dockerfiles and one-command Azure CLI deployment scripts.

If you are building AI applications, you quickly hit a fork in the road: In this guide, I'll walk through how we solved this by building a lightweight FastAPI gateway deployed to Azure Container Apps ACA configured with automated scale-to-zero . The gateway sits between client applications and the underlying Hugging Face Transformers model: /v1/chat/completions and /v1/models . Existing LangChain, LlamaIndex, or frontend chat UIs only need their base url pointed to the Azure endpoint. TextIteratorStreamer for full typewriter streaming in client UIs. Here is the simplified implementation of the OpenAI-compatible streaming endpoint: python import json import asyncio from fastapi import FastAPI, Depends, HTTPException, Security from fastapi.responses import StreamingResponse from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials from pydantic import BaseModel from typing import List, Optional app = FastAPI title="Private Azure AI Gateway" security = HTTPBearer class ChatMessage BaseModel : role: str content: str class ChatCompletionRequest BaseModel : model: str messages: List ChatMessage stream: Optional bool = False temperature: Optional float = 0.7 max tokens: Optional int = 512 @app.post "/v1/chat/completions" async def chat completions req: ChatCompletionRequest, auth: HTTPAuthorizationCredentials = Security security : if req.stream: async def event generator : Stream tokens using Server-Sent Events SSE tokens = "Hello", " from", " self-hosted", " DeepSeek " for token in tokens: chunk = { "id": "chatcmpl-123", "object": "chat.completion.chunk", "choices": {"delta": {"content": token}, "finish reason": None} } yield f"data: {json.dumps chunk }\n\n" await asyncio.sleep 0.05 yield "data: DONE \n\n" return StreamingResponse event generator , media type="text/event-stream" return { "id": "chatcmpl-123", "object": "chat.completion", "choices": { "message": {"role": "assistant", "content": "Sample response"}, "finish reason": "stop" } } In the deployment configuration, the secret sauce is setting min replicas = 0 : { "min replicas": 0, "max replicas": 3, "scale to zero": true, "idle timeout seconds": 300 } When deployed via Azure CLI: az containerapp create \ --name deepseek-gateway \ --resource-group my-ai-rg \ --environment my-aca-env \ --image myacr.azurecr.io/deepseek-gateway:latest \ --target-port 8080 \ --ingress external \ --min-replicas 0 \ --max-replicas 3 \ --scale-rule-name http-rule \ --scale-rule-type http \ --scale-rule-http-concurrency 10 In your existing Python application, you do not need to install custom Hugging Face libraries. Just point the official OpenAI SDK to your Azure endpoint: python from openai import OpenAI client = OpenAI base url="https://deepseek-gateway.azurecontainerapps.io/v1", api key="your-secure-bearer-key" response = client.chat.completions.create model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B", messages= {"role": "user", "content": "Write a Python script to sort a list of dicts."} , stream=True for chunk in response: print chunk.choices 0 .delta.content or "", end="" The complete open-source repository is available on GitHub: 👉 GitHub: faizalbagwan786/azure-model-deployer https://github.com/faizalbagwan786/azure-model-deployer If you want the production-ready boilerplate with pre-configured NVIDIA CUDA 12.1 GPU Dockerfiles, vision support SmolVLM , test suite, and 1-command Azure CLI automated deployment scripts, check out the Gumroad Release $10 https://faizalbagwan.gumroad.com/l/hhtxdv . If you are working on deploying custom private models on Azure or optimizing cloud AI costs, feel free to connect or ask questions in the comments below