I run a small internal speech-synthesis service on an NVIDIA T4 via Azure Container Apps' serverless GPU, with min-replicas=0
. The pitch is that you pay nothing while nobody's using it.
I wanted to know if that's actually true, so I pulled the real numbers out of Azure Cost Management, scoped to just that resource group.
Short answer: yes, it's true. But the biggest line item on my bill wasn't the GPU.
[browser] → [Container App (T4, min=0 → max=1)]
├─ web UI (FastAPI + uvicorn, :8000)
└─ inference (:9880)
Idle for ~5 minutes and it scales to zero replicas. Next request wakes it back up. Cold start is about 5 minutes in my case — a CUDA base image plus model weights.
Measured over 48 days, scoped to one resource group. (Billed in JPY; USD figures are approximate.)
| Service | Per month | Notes |
|---|---|---|
| Container Apps (GPU) | $2–7 | Only charged on days I used it |
| Container Registry | ~$11 | Fixed cost. The biggest line item. |
| Log Analytics | $0 |
Total: around $13/month, including the month where I was actively building and testing.
Here's the daily distribution. Out of 48 days, GPU charges appeared on only 12.
| Charge that day | Days |
|---|---|
| $0 (unused) | 36 |
| $0.01–0.20 | 4 |
| $0.20–0.80 | 5 |
| ~$1.05 | 2 |
| $2.30 | 1 |
A day of real use costs somewhere between four cents and a dollar. An always-on T4 VM of comparable spec runs a few hundred dollars a month. That's close to two orders of magnitude.
The tradeoff is the cold start. For an internal tool where people know to expect a few minutes on first use, that's an easy trade. For a customer-facing endpoint, it isn't.
That one outlier day has a boring explanation. I deployed a new image and temporarily forced the app to stay up so I could read the startup logs:
az containerapp update -n $APP -g $RG --image $ACR.azurecr.io/my-app:2 \
--min-replicas 1 --max-replicas 1
az containerapp update -n $APP -g $RG --min-replicas 0
In a min=0 setup, the expensive mistake is never the GPU's hourly rate. It's leaving min-replicas 1
on after a debugging session. Put the scale-back-down step in your deploy checklist.
The Azure portal showing "Status: Running" means the app exists, not that a replica is up. That confused me early on. The CLI tells the truth:
az containerapp replica list -n $APP -g $RG -o table
az containerapp revision list -n $APP -g $RG \
--query "[?properties.active].properties.runningState" -o tsv
Container Registry was ~$11/month against the GPU's $2–7. Two reasons compounding:
The images are huge. A CUDA base image plus bundled model weights runs several tens of GB per image. I bundle the weights deliberately — down them at startup would add minutes to an already slow cold start.
Old images never leave. I tag every deploy (my-app:1
, :2
, :3
…), and nothing removes the old ones. Storage billing is on the total, so the bill creeps up with every deploy.
The fix is housekeeping. Keep what's deployed plus one for rollback:
az containerapp show -n $APP -g $RG \
--query "properties.template.containers[0].image" -o tsv
az acr repository show-tags -n $ACR --repository my-app --orderby time_desc -o tsv
for t in 1 2 3 4 5 6 7 8; do
az acr repository delete -n $ACR --image my-app:$t --yes
done
I went from 10 images down to 2. Storage billing catches up over a few hours, via the registry's garbage collection.
If you're evaluating serverless GPU, budget for registry storage. It's the line item nobody warns you about, and for a low-traffic workload it can quietly become your largest one.
--min-replicas 0
.Getting from zero to a working deployment took considerably longer than it should have. Pinning torch
against transformers
' requirements, a 504 caused by the order services start in, and — on Windows — az acr build
crashing the CLI on a character encoding issue mid-build. None of that is in the docs.
I wrote all of it up, including the full deployment walkthrough, here:
Zero-Idle GPU: Running Inference on Azure Container Apps
Happy to answer questions in the comments.