From A10 to M60: An Architect's Journey into Azure GPU VM Sizing for Kubernetes Inference Workloads

An architect at a company deploying a Visual Element Detection (VED) service on Azure Kubernetes Service (AKS) encountered an unexpected constraint when the target region did not offer the NVads_A10_v5 GPU VMs used in the source region. This forced a deep dive into Azure GPU VM families, naming conventions, and workload characteristics to select an appropriate alternative. The team analyzed the PyTorch inference workload—under 200 MB model size, ~2000x2000 image resolution, 5-7 requests/sec throughput—and learned to decode Azure's VM naming scheme, understanding that NV-series VMs are for visualization and lightweight inference, NC-series for AI workloads, and ND-series for training, while also discovering that NVads_A10_v5 instances may provide only a fraction of the GPU (e.g., 1/6th of an A10).

How an unexpected regional constraint forced us to deeply understand Azure GPU VM families, naming conventions, and workload fit. As architects, we often assume that infrastructure decisions are straightforward: "The workload is already running successfully in Region A. Let's deploy the same Kubernetes workload in Region B." That's exactly what we thought. Our workload consisted of a Visual Element Detection VED service hosted on Kubernetes. The application uses a PyTorch model to analyze images and detect various visual elements in an image file. The service was already running successfully on a node pool backed by Azure's NVads A10 v5 GPU VMs. Then we hit an unexpected challenge. The target region did not offer NVads A10 v5 instances. What looked like a simple deployment exercise became a deep dive into Azure GPU virtual machine families, GPU architectures, VM naming conventions, and workload characteristics. This article shares what I learned in the hope that it helps others who find themselves evaluating Azure GPU SKUs for AI inference workloads. I am relatively new to the world of MLOps, Model deployments, GPU Workloads etc and equally interested and excited to learn more on this front. Before discussing VM selection, let's understand the workload characteristics: Model Type : PyTorch Model Size : less than 200 MB .pth Image Resolution : ~2000 x 2000 Expected Throughput: 5-7 requests/sec Platform : AKS Kubernetes Workload Type : Inference only This is important because GPU sizing should always start from the workload and not from the VM catalog. Many engineers first encounter Azure GPU machines through names like: NV12s v3 NV6ads A10 v5 NC4as T4 v3 ND96isr H100 v5 The naming can be intimidating. The first breakthrough was understanding that Azure organizes GPU VMs into three primary families: N-Series ├── NV ├── NC └── ND NV-series VMs are designed primarily for: Examples: NV12s v32 NV24s v33 NV6ads A10 v54 Typical GPUs: NVIDIA Tesla M60 NVIDIA A10 AMD MI25 Historically, these SKUs were optimized for graphics workloads, although many organizations now use them for lightweight AI inference workloads. NC-series VMs are optimized for: Examples: NC4as T4 v32 NC8as T4 v33 NC A100 v44 NCads H100 v5 Typical GPUs: T4 V100 A100 H100 For pure AI workloads, NC is often the most natural fit. ND-series is designed for: Examples: ND A100 v42 ND H100 v5 If you're training LLMs, ND is your friend. If you're serving a 200 MB inference model, ND is probably overkill. One of the biggest learnings from this exercise was understanding Azure's VM naming scheme. Let's decode the VM we were already using: NV6ads A10 v5 Breaking it apart: | Component | Meaning | |---|---| | NV | Visualization family | | 6 | vCPU count | | a | AMD processor | | d | Local temporary disk | | s | Premium SSD support | | A10 | NVIDIA A10 GPU | | v5 | Generation 5 | Once you learn the modifiers, every Azure VM becomes easier to understand. Common Suffixes You'll Encounter as - NC4as T4 v3 ads - NV6ads A10 v5 adms - NV36adms A10 v5 These memory-optimized variants generally provide significantly more RAM than their standard counterparts. Common Letters we will usually see - | Letter | Meaning | |---|---| | a | AMD CPU | | b | Higher storage bandwidth | | d | Local/temp disk | | e | Confidential/encrypted capabilities | | m | Memory optimized | | n | Network optimized | | p | ARM processor | | r | RDMA / InfiniBand | | s | Premium SSD support | One detail many people miss about NVads A10 v5 is that you may not receive a full GPU. For example: NV6ads A10 v5 provides: 1/6th of an NVIDIA A10 ≈ 4 GB VRAM Azure uses GPU partitioning to expose slices of the physical A10 GPU. Conceptually: php A10 GPU 24 GB ├─ VM1 - 4 GB ├─ VM2 - 4 GB ├─ VM3 - 4 GB ├─ VM4 - 4 GB ├─ VM5 - 4 GB └─ VM6 - 4 GB This creates cost-effective GPU options, especially for inference workloads. Because the A10-based SKUs were unavailable in our target region, we began evaluating: NV12s v3, NV24s v3 and NV48s v3 These machines use the much older - NVIDIA Tesla M60 GPU. The key specifications: | SKU | GPUs | GPU Memory | |---|---|---| | NV12s v3 | 1 × M60 | 16 GB | | NV24s v3 | 2 × M60 | 32 GB | | NV48s v3 | 4 × M60 | 64 GB | At first glance, moving from an A10 to an M60 looked risky. However, after analyzing the workload, we realized something important: The model itself was only less than 200MB so memory wasn't the challenge. The real challenge was 2000 x 2000 image processing which drives compute consumption much more than model size. A common temptation in infrastructure planning is: "Let's buy the biggest VM and move on." We resisted that temptation. For our workload: Inference only the most logical starting point became: NV12s v3 Why? 16 GB GPU memory Sufficient CPU capacity Lower cost Easier horizontal scaling in Kubernetes Rather than using: 1 × NV48s v3, we preferred the Kubernetes-native approach: If you're evaluating Azure GPU machines for inference workloads, remember: Understand the family first NV = Visualization NC = Compute ND = Deep Learning Understand the suffixes as = AMD + Premium SSD ads = AMD + Local Disk + Premium SSD adms = AMD + Local Disk + Memory Optimized + Premium SSD Don't focus only on GPU names A larger GPU doesn't automatically mean you need a larger VM. Consider: Start with measurement using Benchmark: nvidia-smi Monitor: Then scale based on evidence. What began as a regional availability issue became a valuable learning opportunity. We started with a simple question: "What should we use if NVads A10 v5 isn't available?" We ended up gaining a much deeper understanding of: As architects, these are the moments that help us move beyond simply selecting SKUs and toward making informed infrastructure decisions based on workload characteristics. Sometimes the best architecture lessons come from constraints rather than from choice. Have you had to migrate inference workloads between Azure regions and deal with GPU availability differences? I'd love to hear what SKUs and strategies worked for your teams.