{"slug": "from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes", "title": "From A10 to M60: An Architect's Journey into Azure GPU VM Sizing for Kubernetes Inference Workloads", "summary": "An architect at a company deploying a Visual Element Detection (VED) service on Azure Kubernetes Service (AKS) encountered an unexpected constraint when the target region did not offer the NVads_A10_v5 GPU VMs used in the source region. This forced a deep dive into Azure GPU VM families, naming conventions, and workload characteristics to select an appropriate alternative. The team analyzed the PyTorch inference workload—under 200 MB model size, ~2000x2000 image resolution, 5-7 requests/sec throughput—and learned to decode Azure's VM naming scheme, understanding that NV-series VMs are for visualization and lightweight inference, NC-series for AI workloads, and ND-series for training, while also discovering that NVads_A10_v5 instances may provide only a fraction of the GPU (e.g., 1/6th of an A10).", "body_md": "How an unexpected regional constraint forced us to deeply understand Azure GPU VM families, naming conventions, and workload fit.\n\nAs architects, we often assume that infrastructure decisions are straightforward:\n\n\"The workload is already running successfully in Region A. Let's deploy the same Kubernetes workload in Region B.\"\n\n__That's exactly what we thought.__\n\nOur workload consisted of a **Visual Element Detection (VED)** service hosted on Kubernetes. The application uses a PyTorch model to analyze images and detect various visual elements in an image file. The service was already running successfully on a node pool backed by Azure's NVads_A10_v5 GPU VMs.\n\nThen we hit an unexpected challenge.\n\nThe target region did not offer NVads_A10_v5 instances.\n\nWhat looked like a simple deployment exercise became a deep dive into Azure GPU virtual machine families, GPU architectures, VM naming conventions, and workload characteristics.\n\nThis article shares what I learned in the hope that it helps others who find themselves evaluating Azure GPU SKUs for AI inference workloads. I am relatively new to the world of MLOps, Model deployments, GPU Workloads etc and equally interested and excited to learn more on this front.\n\nBefore discussing VM selection, let's understand the workload characteristics:\n\n```\nModel Type        : PyTorch\nModel Size        : less than 200 MB (.pth)\nImage Resolution  : ~2000 x 2000\nExpected Throughput: 5-7 requests/sec\nPlatform          : AKS (Kubernetes)\nWorkload Type     : Inference only\n```\n\nThis is important because GPU sizing should always start from the workload and not from the VM catalog.\n\nMany engineers first encounter Azure GPU machines through names like:\n\n```\nNV12s_v3\nNV6ads_A10_v5\nNC4as_T4_v3\nND96isr_H100_v5\n```\n\nThe naming can be intimidating.\n\nThe first breakthrough was understanding that Azure organizes GPU VMs into three primary families:\n\n```\nN-Series\n├── NV\n├── NC\n└── ND\n```\n\nNV-series VMs are designed primarily for:\n\nExamples:\n\nNV12s_v32\n\nNV24s_v33\n\nNV6ads_A10_v54\n\n**Typical GPUs:**\n\nNVIDIA Tesla M60\n\nNVIDIA A10\n\nAMD MI25\n\nHistorically, these SKUs were optimized for graphics workloads, although many organizations now use them for lightweight AI inference workloads.\n\nNC-series VMs are optimized for:\n\nExamples:\n\nNC4as_T4_v32\n\nNC8as_T4_v33\n\nNC_A100_v44\n\nNCads_H100_v5\n\n**Typical GPUs:**\n\nT4\n\nV100\n\nA100\n\nH100\n\n*For pure AI workloads, NC is often the most natural fit.*\n\nND-series is designed for:\n\nExamples:\n\nND_A100_v42\n\nND_H100_v5\n\nIf you're training LLMs, ND is your friend.\n\nIf you're serving a 200 MB inference model, ND is probably overkill.\n\nOne of the biggest learnings from this exercise was understanding Azure's VM naming scheme.\n\nLet's decode the VM we were already using:\n\n**NV6ads_A10_v5**\n\nBreaking it apart:\n\n| Component | Meaning |\n|---|---|\n| NV | Visualization family |\n| 6 | vCPU count |\n| a | AMD processor |\n| d | Local temporary disk |\n| s | Premium SSD support |\n| A10 | NVIDIA A10 GPU |\n| v5 | Generation 5 |\n\nOnce you learn the modifiers, every Azure VM becomes easier to understand.\n\nCommon Suffixes You'll Encounter\n\nas - NC4as_T4_v3\n\nads - NV6ads_A10_v5\n\nadms - NV36adms_A10_v5\n\nThese memory-optimized variants generally provide significantly more RAM than their standard counterparts.\n\nCommon Letters we will usually see -\n\n| Letter | Meaning |\n|---|---|\n| a | AMD CPU |\n| b | Higher storage bandwidth |\n| d | Local/temp disk |\n| e | Confidential/encrypted capabilities |\n| m | Memory optimized |\n| n | Network optimized |\n| p | ARM processor |\n| r | RDMA / InfiniBand |\n| s | Premium SSD support |\n\nOne detail many people miss about NVads_A10_v5 is that you may not receive a full GPU.\n\nFor example:\n\nNV6ads_A10_v5 provides: 1/6th of an NVIDIA A10 ≈ 4 GB VRAM\n\nAzure uses GPU partitioning to expose slices of the physical A10 GPU.\n\nConceptually:\n\n``` php\nA10 GPU (24 GB)\n ├─ VM1 -> 4 GB\n ├─ VM2 -> 4 GB\n ├─ VM3 -> 4 GB\n ├─ VM4 -> 4 GB\n ├─ VM5 -> 4 GB\n └─ VM6 -> 4 GB\n``\n```\n\nThis creates cost-effective GPU options, especially for inference workloads.\n\nBecause the A10-based SKUs were unavailable in our target region, we began evaluating:\n\nNV12s_v3, NV24s_v3 and NV48s_v3\n\nThese machines use the much older - NVIDIA Tesla M60 GPU.\n\nThe key specifications:\n\n| SKU | GPUs | GPU Memory |\n|---|---|---|\n| NV12s_v3 | 1 × M60 | 16 GB |\n| NV24s_v3 | 2 × M60 | 32 GB |\n| NV48s_v3 | 4 × M60 | 64 GB |\n\nAt first glance, moving from an A10 to an M60 looked risky.\n\nHowever, after analyzing the workload, we realized something important:\n\nThe model itself was only less than 200MB so memory wasn't the challenge.\n\nThe real challenge was 2000 x 2000 image processing which drives compute consumption much more than model size.\n\nA common temptation in infrastructure planning is:\n\n\"Let's buy the biggest VM and move on.\"\n\nWe resisted that temptation.\n\nFor our workload:\n\nInference only\n\nthe most logical starting point became:\n\nNV12s_v3\n\nWhy?\n\n16 GB GPU memory\n\nSufficient CPU capacity\n\nLower cost\n\nEasier horizontal scaling in Kubernetes\n\nRather than using:\n\n1 × NV48s_v3, we preferred the Kubernetes-native approach:\n\nIf you're evaluating Azure GPU machines for inference workloads, remember:\n\nUnderstand the family first\n\n**NV** = Visualization\n\n**NC**= Compute\n\n**ND**= Deep Learning\n\nUnderstand the suffixes\n\n**as** = AMD + Premium SSD\n\n**ads** = AMD + Local Disk + Premium SSD\n\n**adms** = AMD + Local Disk + Memory Optimized + Premium SSD\n\nDon't focus only on GPU names\n\nA larger GPU doesn't automatically mean you need a larger VM.\n\nConsider:\n\n**Start with measurement** using Benchmark:\n\n*nvidia-smi*\n\nMonitor:\n\nThen scale based on evidence.\n\nWhat began as a regional availability issue became a valuable learning opportunity.\n\nWe started with a simple question:\n\n*\"What should we use if NVads_A10_v5 isn't available?\"*\n\nWe ended up gaining a much deeper understanding of:\n\nAs architects, these are the moments that help us move beyond simply selecting SKUs and toward making informed infrastructure decisions based on workload characteristics.\n\nSometimes the best architecture lessons come from constraints rather than from choice.\n\n*Have you had to migrate inference workloads between Azure regions and deal with GPU availability differences? I'd love to hear what SKUs and strategies worked for your teams.*", "url": "https://wpnews.pro/news/from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes", "canonical_source": "https://dev.to/archcode01/from-a10-to-m60-an-architects-journey-into-azure-gpu-vm-sizing-for-kubernetes-inference-workloads-4hjj", "published_at": "2026-07-15 12:45:45+00:00", "updated_at": "2026-07-15 12:59:41.244946+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "developer-tools", "mlops"], "entities": ["Azure", "Kubernetes", "NVIDIA A10", "NVIDIA Tesla M60", "PyTorch", "AKS", "NVads_A10_v5", "Visual Element Detection"], "alternates": {"html": "https://wpnews.pro/news/from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes", "markdown": "https://wpnews.pro/news/from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes.md", "text": "https://wpnews.pro/news/from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes.txt", "jsonld": "https://wpnews.pro/news/from-a10-to-m60-an-architect-s-journey-into-azure-gpu-vm-sizing-for-kubernetes.jsonld"}}