How an unexpected regional constraint forced us to deeply understand Azure GPU VM families, naming conventions, and workload fit.
As architects, we often assume that infrastructure decisions are straightforward:
"The workload is already running successfully in Region A. Let's deploy the same Kubernetes workload in Region B."
That's exactly what we thought.
Our workload consisted of a Visual Element Detection (VED) service hosted on Kubernetes. The application uses a PyTorch model to analyze images and detect various visual elements in an image file. The service was already running successfully on a node pool backed by Azure's NVads_A10_v5 GPU VMs.
Then we hit an unexpected challenge.
The target region did not offer NVads_A10_v5 instances.
What looked like a simple deployment exercise became a deep dive into Azure GPU virtual machine families, GPU architectures, VM naming conventions, and workload characteristics.
This article shares what I learned in the hope that it helps others who find themselves evaluating Azure GPU SKUs for AI inference workloads. I am relatively new to the world of MLOps, Model deployments, GPU Workloads etc and equally interested and excited to learn more on this front.
Before discussing VM selection, let's understand the workload characteristics:
Model Type : PyTorch
Model Size : less than 200 MB (.pth)
Image Resolution : ~2000 x 2000
Expected Throughput: 5-7 requests/sec
Platform : AKS (Kubernetes)
Workload Type : Inference only
This is important because GPU sizing should always start from the workload and not from the VM catalog.
Many engineers first encounter Azure GPU machines through names like:
NV12s_v3
NV6ads_A10_v5
NC4as_T4_v3
ND96isr_H100_v5
The naming can be intimidating.
The first breakthrough was understanding that Azure organizes GPU VMs into three primary families:
N-Series
├── NV
├── NC
└── ND
NV-series VMs are designed primarily for:
Examples:
NV12s_v32
NV24s_v33
NV6ads_A10_v54
Typical GPUs:
NVIDIA Tesla M60
NVIDIA A10
AMD MI25
Historically, these SKUs were optimized for graphics workloads, although many organizations now use them for lightweight AI inference workloads.
NC-series VMs are optimized for:
Examples:
NC4as_T4_v32
NC8as_T4_v33
NC_A100_v44
NCads_H100_v5
Typical GPUs:
T4
V100
A100
H100
For pure AI workloads, NC is often the most natural fit.
ND-series is designed for:
Examples:
ND_A100_v42
ND_H100_v5
If you're training LLMs, ND is your friend.
If you're serving a 200 MB inference model, ND is probably overkill.
One of the biggest learnings from this exercise was understanding Azure's VM naming scheme.
Let's decode the VM we were already using:
NV6ads_A10_v5
Breaking it apart:
| Component | Meaning |
|---|---|
| NV | Visualization family |
| 6 | vCPU count |
| a | AMD processor |
| d | Local temporary disk |
| s | Premium SSD support |
| A10 | NVIDIA A10 GPU |
| v5 | Generation 5 |
Once you learn the modifiers, every Azure VM becomes easier to understand.
Common Suffixes You'll Encounter
as - NC4as_T4_v3
ads - NV6ads_A10_v5
adms - NV36adms_A10_v5
These memory-optimized variants generally provide significantly more RAM than their standard counterparts.
Common Letters we will usually see -
| Letter | Meaning |
|---|---|
| a | AMD CPU |
| b | Higher storage bandwidth |
| d | Local/temp disk |
| e | Confidential/encrypted capabilities |
| m | Memory optimized |
| n | Network optimized |
| p | ARM processor |
| r | RDMA / InfiniBand |
| s | Premium SSD support |
One detail many people miss about NVads_A10_v5 is that you may not receive a full GPU.
For example:
NV6ads_A10_v5 provides: 1/6th of an NVIDIA A10 ≈ 4 GB VRAM
Azure uses GPU partitioning to expose slices of the physical A10 GPU.
Conceptually:
A10 GPU (24 GB)
├─ VM1 -> 4 GB
├─ VM2 -> 4 GB
├─ VM3 -> 4 GB
├─ VM4 -> 4 GB
├─ VM5 -> 4 GB
└─ VM6 -> 4 GB
``
This creates cost-effective GPU options, especially for inference workloads.
Because the A10-based SKUs were unavailable in our target region, we began evaluating:
NV12s_v3, NV24s_v3 and NV48s_v3
These machines use the much older - NVIDIA Tesla M60 GPU.
The key specifications:
| SKU | GPUs | GPU Memory |
|---|---|---|
| NV12s_v3 | 1 × M60 | 16 GB |
| NV24s_v3 | 2 × M60 | 32 GB |
| NV48s_v3 | 4 × M60 | 64 GB |
At first glance, moving from an A10 to an M60 looked risky.
However, after analyzing the workload, we realized something important:
The model itself was only less than 200MB so memory wasn't the challenge.
The real challenge was 2000 x 2000 image processing which drives compute consumption much more than model size.
A common temptation in infrastructure planning is:
"Let's buy the biggest VM and move on."
We resisted that temptation.
For our workload:
Inference only
the most logical starting point became:
NV12s_v3
Why?
16 GB GPU memory
Sufficient CPU capacity
Lower cost
Easier horizontal scaling in Kubernetes
Rather than using:
1 × NV48s_v3, we preferred the Kubernetes-native approach:
If you're evaluating Azure GPU machines for inference workloads, remember:
Understand the family first
NV = Visualization
NC= Compute
ND= Deep Learning
Understand the suffixes
as = AMD + Premium SSD
ads = AMD + Local Disk + Premium SSD
adms = AMD + Local Disk + Memory Optimized + Premium SSD
Don't focus only on GPU names
A larger GPU doesn't automatically mean you need a larger VM.
Consider:
Start with measurement using Benchmark:
nvidia-smi
Monitor:
Then scale based on evidence.
What began as a regional availability issue became a valuable learning opportunity.
We started with a simple question:
"What should we use if NVads_A10_v5 isn't available?"
We ended up gaining a much deeper understanding of:
As architects, these are the moments that help us move beyond simply selecting SKUs and toward making informed infrastructure decisions based on workload characteristics.
Sometimes the best architecture lessons come from constraints rather than from choice.
Have you had to migrate inference workloads between Azure regions and deal with GPU availability differences? I'd love to hear what SKUs and strategies worked for your teams.