{"slug": "kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization", "title": "Kubernetes Rightsizing: A Practical Workflow for CPU and Memory Optimization", "summary": "Kubernetes rightsizing matches pod resource requests to actual usage, closing an overprovisioning gap where 69% of requested CPU goes unused, according to the 2026 Cast AI report. A five-step workflow—Observe, Set Requests, Set Limits, Verify, Repeat—can reduce compute costs by 40-70% without code changes. Kubernetes 1.33+ supports zero-downtime in-place resizing, and GPU waste, averaging 5% utilization, can be cut by 70% via fractional sharing.", "body_md": "Kubernetes rightsizing is the practice of matching pod CPU and memory requests and limits to actual usage, so you stop paying for capacity you never use without risking throttling or OOM kills. Most clusters show 8% average CPU utilization: pods consume only a small fraction of what they allocate. The 2026 Cast AI report quantifies the gap: 69% of requested CPU across production clusters goes entirely unused. That is the overprovisioning gap that rightsizing closes.\n\n## Key Takeaways\n\n**CPU is wasted by default:** Average CPU utilization across production Kubernetes clusters is 8%; 69% of requested CPU goes entirely unused ([Cast AI 2026 State of Kubernetes Optimization Report](https://cast.ai/reports/state-of-kubernetes-optimization/)).**Rightsizing is the safest cost lever:** Correcting resource requests requires no code changes, no replica adjustments, and no architectural redesign only configuration fixes.**Follow the five-step workflow:** Observe real usage (2+ weeks) → Set CPU requests at p95 typical → Set limits to protect neighbors → Verify with throttling and OOM checks → Repeat as workloads evolve.**Zero-downtime resizing is available:** Kubernetes 1.33+ supports in-place pod resizing (InPlacePodVerticalScaling, beta in 1.33, GA in 1.35); CPU adjustments apply without a container restart.**GPU waste is the most expensive gap:** Average GPU utilization is 5%; fractional GPU sharing via time-slicing, MIG, or NVIDIA MPS can cut GPU costs by 70% or more without changing model code.**Expected savings:** 40–70% compute cost reduction is realistic for clusters with no previous rightsizing; automated rightsizing (Cast AI Workload Autoscaler) has delivered up to 93% utilization improvement.\n\n## What Rightsizing Is and Why It Is the Safest Lever\n\nRequests define the CPU and memory Kubernetes *reserves* on a node for a pod. Limits define how much the pod can consume before it gets throttled (CPU) or killed (memory). The scheduler uses requests to place pods. Nodes fill up based on requests, not actual usage.\n\nThe gap between those two numbers is where your money disappears. The Cast AI 2026 State of Kubernetes Optimization Report measured tens of thousands of production clusters across AWS, GCP, and Azure. Average CPU utilization: 8%. Average memory utilization: 20%. CPU overprovisioning sits at 69% (up from 40% year over year). Memory overprovisioning is 79%.\n\nThose are two distinct measurements. The 8% figure is average CPU utilization: how much of reserved capacity pods actually consume at runtime. The 69% figure is the overprovisioning ratio: how much of CPU requests is never consumed across production clusters. Together, they confirm that most clusters provision many times more CPU than their workloads actually need.\n\nNo autoscaler can fix this at the node level if pods hold oversized requests. Node autoscalers see a “full” cluster and add nodes. The waste scales with you.\n\nRightsizing is the safest lever because it reduces waste without changing what your application does. You are not modifying code, not changing replicas, not touching network policy. You are correcting a configuration error that most teams introduced when they set resources by gut feel during initial deployment and never revisited.\n\n## The Rightsizing Workflow\n\nThe Rightsizing Workflow is a five-step cycle: Observe, Set Requests to Typical, Set Limits to Protect, Verify, Repeat. It applies whether you are doing this manually, with VPA, or with an automated platform. The cycle does not end after the first pass.\n\n### Observe\n\nStart with real usage data. Two weeks minimum. One week misses weekly batch jobs. One day misses end-of-month spikes. The longer the observation window, the more accurate the recommendation.\n\nFor a quick baseline, run these two commands:\n\n```\n# Check actual usage vs requests\nkubectl top pod -n <namespace> --containers\n\n# Check what's allocated\nkubectl describe pod <pod-name> -n <namespace> | grep -A 10 \"Requests:\"\n```\n\nThe first command shows live usage. The second shows what Kubernetes reserved. If the gap is large, rightsizing will have a meaningful impact.\n\nFor statistical sizing over longer windows, use these PromQL queries directly against Prometheus:\n\n```\n# Compute p95 CPU usage over the past 2 weeks (for requests sizing)\nquantile_over_time(0.95,\n  rate(container_cpu_usage_seconds_total{container!=\"\"}[5m])[2w:5m]\n) by (namespace, pod, container)\n\n# Compute p99 memory usage over 2 weeks (for limits sizing)\nquantile_over_time(0.99,\n  container_memory_working_set_bytes{container!=\"\"}[2w:5m]\n) by (namespace, pod, container)\n```\n\nThese queries work best with 2+ weeks of Prometheus history. For longer retention, use Thanos or Cortex. KRR is designed for exactly this use case and queries those backends natively.\n\nFor CPU throttling, you need Prometheus. Throttling is invisible at the pod level: no log, no event, no status change. The container keeps running, but latency increases silently. Detect it with:\n\n```\nsum(rate(container_cpu_cfs_throttled_periods_total[5m])) by (pod, container) /\nsum(rate(container_cpu_cfs_periods_total[5m])) by (pod, container) > 0.25\n```\n\nA result above 0.25 means more than 25% of CPU scheduling periods are throttled. That is a performance problem hiding in your metrics.\n\n### Set Requests to Typical\n\nRequests should reflect typical workload behavior, not peak. Use the p95 or p99 of observed CPU usage over your observation window. Sizing to the absolute peak means you overprovision for rare spikes that last seconds.\n\nHere is what that looks like in practice. A service that was provisioned by hand at deployment:\n\n```\n# Before (over-provisioned)\nresources:\n  requests:\n    cpu: \"2\"\n    memory: \"4Gi\"\n  limits:\n    cpu: \"4\"\n    memory: \"8Gi\"\n```\n\nAfter observing p95 usage over two weeks:\n\n```\n# After (rightsized via p95 observed usage)\nresources:\n  requests:\n    cpu: \"200m\"\n    memory: \"512Mi\"\n  limits:\n    cpu: \"600m\"\n    memory: \"1Gi\"\n```\n\nThe CPU request dropped from 2000m to 200m. The memory request dropped from 4Gi to 512Mi. The node scheduler now sees this pod as ten times smaller, freeing capacity for other workloads or smaller nodes.\n\n### Set Limits to Protect\n\nLimits exist to prevent one pod from consuming all node capacity and affecting neighbors. For CPU, set limits to 2-3x the request. CPU is compressible: exceeding the limit throttles the container, it does not kill it. For memory, be more conservative.\n\nFor memory-sensitive workloads, set request equal to limit (Guaranteed QoS). This prevents eviction under node memory pressure. For workloads where occasional bursting is acceptable, set request below limit (Burstable QoS). Never omit memory limits on multi-tenant clusters.\n\nFor CPU on most production workloads, consider omitting limits entirely and relying on requests for scheduling. CPU limits with tight settings cause more throttling incidents than the overprovisioning they prevent.\n\nJVM workloads need special attention. The JVM heap size is separate from the container memory limit. Java applications will try to expand the heap up to the limit, which means your container memory request should account for: JVM heap (-Xmx) plus roughly 25% overhead for metaspace, thread stacks, and off-heap buffers. If the limit is too tight, you get OOMKills. If requests are too loose, you pay for memory the JVM never uses. Set -Xmx to 75% of the container memory limit as a starting point.\n\n### Verify\n\nAfter applying new requests and limits, watch for two failure signals. First, check the Prometheus throttling query again 24 hours after the change. If throttling increased significantly, the new CPU request is too low. Second, watch for OOMKilled events:\n\n```\nkubectl get events -n <namespace> --field-selector reason=OOMKilling\nkubectl describe pod <pod-name> -n <namespace> | grep -i oom\n```\n\nA single OOMKill after rightsizing means the memory request landed too low. Increase the memory request and limit, then re-observe.\n\nIf p99 latency increases or error rates rise after applying new limits, the cause is likely CPU throttling (limits too tight) or OOM kills (memory limits too tight). Rollback immediately:\n\n```\nkubectl set resources deployment <name> -c <container> --requests=cpu=<original> --limits=cpu=<original>\n```\n\nFor automated platforms, temporarily suspend rightsizing for that workload until you identify the safe limit floor.\n\n### Repeat\n\nWorkloads change. A new feature ships. Traffic patterns shift. A library version increases baseline memory footprint. Static rightsizing decays. Treat it as a recurring process, not a one-time task.\n\nFor manual processes: set a calendar reminder every two to four weeks, or after every major release. For automated platforms: Cast AI’s Workload Autoscaler refreshes recommendations every 30 minutes and reacts immediately to OOM events or usage spikes exceeding 50% above the current recommendation.\n\n## CPU vs Memory Failure Modes\n\nCPU and memory fail differently. Knowing the difference changes how you set limits and how urgently you respond to threshold breaches.\n\n### Throttling vs OOMKilled\n\nCPU is a *compressible* resource. When a container exceeds its CPU limit, the kernel throttles it. The process keeps running. Your application slows down. This is silent: no Kubernetes event, no log entry, no status change. Latency increases, timeouts creep up, SLOs degrade. You may not connect the cause to a CPU limit for days.\n\nMemory is *incompressible*. When a container exceeds its memory limit, the kernel kills it immediately with an OOMKill (exit code 137). Kubernetes restarts the container. Depending on your readiness probe configuration, this causes a visible service interruption. On StatefulSets or pods with warm caches, each restart is expensive.\n\nSee the detailed breakdown of OOMKilled behavior and exit codes: [OOMKilled Exit Code 137 explained](https://cast.ai/blog/oomkilled-exit-code-137/).\n\nOne production cluster in the 2026 report recorded 40-50 OOM kills per 30-minute window (measured across a representative production cluster with generous resource padding; the interval is the platform’s 30-minute recommendation cycle). After applying recommendations with OOM detection overhead added automatically, OOM kills dropped to near zero while provisioned CPUs were cut by roughly half.\n\n#### How to Set Each\n\nThe sizing principles follow the workflow steps above. For CPU, use p95 observed usage for requests and 2-3x the request for limits, or omit limits entirely to avoid silent throttling. For memory, use p99 or peak for requests; set limits equal to the request for Guaranteed QoS on critical workloads, or up to 1.5x the request for Burstable QoS on batch workloads. JVM services require the additional heap headroom calculation described in the Set Limits to Protect step.\n\n## Manual vs VPA vs Automated\n\nThree approaches exist for getting recommendations into production. Each has a different cost/benefit profile.\n\n| Approach | Detection | Application | Downtime | Accuracy | Ops Burden |\n|---|---|---|---|---|---|\n| Manual | Engineer runs kubectl top / PromQL | PR + deployment | Rolling restart | Low (point-in-time) | High |\n| VPA (Recreate) | VPA Recommender (8-day window) | Updater evicts pods | Pod eviction + restart | Medium | Low |\n| Automated (Cast AI) | Continuous, every 30 min + anomaly detection | In-place or deferred | None (in-place) or controlled | High (OOM-aware) | Very low |\n\n### Goldilocks\n\nGoldilocks (Fairwinds) wraps VPA in recommendation-only mode. It creates a VPA object per Deployment in namespaces you label with `goldilocks.fairwinds.com/enabled=true`\n\n. No pods change. You view recommendations in a web dashboard that shows both Guaranteed QoS (request equals limit) and Burstable QoS (request below limit) options.\n\nSetup: install VPA first, then Goldilocks, then label the namespace. The VPA Updater mode stays Off throughout. This makes Goldilocks safe to run alongside production workloads with zero risk of pod changes. The trade-off: you still apply changes manually.\n\n### KRR\n\nKRR (Robusta) queries Prometheus directly, bypassing the VPA API entirely. This matters because VPA retains only 8 days of history. KRR can query Thanos, Cortex, or Mimir for longer windows, giving more accurate recommendations for workloads with weekly or monthly traffic patterns.\n\n```\npip install robusta-krr\nkrr simple --prometheus-url http://prometheus:9090\n```\n\nKRR ships with customizable strategies: percentile (default p99) for CPU, peak for memory. No admission webhook means it is safe to run on any cluster without side effects. Like Goldilocks, it produces recommendations. You apply them.\n\n### VPA Modes\n\nVPA ships with five update modes. Each has different disruption characteristics:\n\n**Off:** Recommendations only. No pods changed. Safe starting point for any cluster.**Initial:** Sets requests at pod creation only. Existing pods unchanged. Low risk.**Recreate:** The Updater evicts and recreates pods to apply new values. Disrupts StatefulSets with caches, JIT-warmed code, or leader elections. Use with caution.**InPlaceOrRecreate:** Attempts live resize first, falls back to eviction. Requires Kubernetes 1.33+ (InPlacePodVerticalScaling beta) or 1.35+ for GA and default-on behavior.**InPlace:** Never evicts. Alpha in VPA 1.7.0. Requires Kubernetes 1.33+.\n\nVPA limitations to know before enabling Recreate mode: VPA needs 24-48 hours of data minimum and retains only 8 days. When HPA is also active, VPA lowering requests causes HPA utilization percentages to rise, which can trigger unintended horizontal scale-out. Run HPA off CPU utilization and VPA together only with care.\n\nThe mitigation: configure HPA to scale on external or custom metrics such as request rate or queue depth rather than CPU utilization when VPA is active. That way, VPA adjusts requests without affecting HPA’s scaling trigger. KEDA is a strong option here. It scales on message queue length, HTTP request rate, or any Prometheus metric, avoiding the feedback loop entirely.\n\n### Platforms That Act\n\nThe Cast AI Workload Autoscaler moves beyond recommendation into continuous automated application. It supports Deployments, StatefulSets, DaemonSets, Rollouts, ReplicaSets, and CronJobs. Recommendations regenerate every 30 minutes. If usage spikes more than 50% above the current recommendation, regeneration triggers immediately.\n\nOOM detection runs by monitoring pod container statuses. When a container OOMKills, Cast AI adds overhead automatically to the memory recommendation before the next apply cycle. This is the mechanism behind the 40-50 OOM kills per interval dropping to near zero in the 2026 report cluster data.\n\nApply modes are Immediate (changes applied as soon as a recommendation is ready) and Deferred (changes queued for the next maintenance window or pod restart). High-availability mode with two replicas is available from v0.30.0.\n\n#### In-Place vs Restart\n\nThe in-place pod resize feature eliminates the need to evict and recreate pods when changing requests. The timeline: alpha in Kubernetes 1.27, beta in Kubernetes 1.33 (May 2025), GA and default-on in Kubernetes 1.35 (December 2025). Cast AI Workload Autoscaler supports in-place resizing from v0.53.0.\n\nFor clusters still below 1.33, restart-based application remains the default. PodDisruptionBudgets (covered below) limit the blast radius.\n\n## Rightsizing Without Downtime\n\nThe most common objection to rightsizing in production is disruption risk. Two mechanisms address this directly.\n\n### In-Place Pod Resize\n\nIn-place pod resizing (InPlacePodVerticalScaling) lets the kubelet update a running container’s CPU and memory resources without restarting it. The kernel adjusts the cgroup limits live. Without evictions, restarts, or service interruptions.\n\nThe feature timeline: alpha in Kubernetes 1.27, beta in Kubernetes 1.33 (May 2025), GA and default-on in Kubernetes 1.35 (December 2025). On clusters running 1.33+, you can use InPlace or InPlaceOrRecreate modes with the feature gate enabled. Whereas on 1.35+, it is on by default with no gate required.\n\nOn managed clusters (EKS, GKE, AKS), check whether your control plane version exposes InPlacePodVerticalScaling. Managed versions often lag upstream by 2-4 months. Verify with: `kubectl get node -o jsonpath='{.items[0].status.nodeInfo.kubeletVersion}'`\n\nand your cloud provider’s feature gate documentation.\n\nThis is the biggest operational change for rightsizing since VPA shipped. StatefulSets with warm caches, JVM workloads with compiled code, and leader-elected controllers all benefit immediately because pod churn disappears.\n\nControl per-container resize behavior with the `resizePolicy`\n\nfield:\n\n```\nspec:\n  containers:\n  - name: myapp\n    resizePolicy:\n    - resourceName: cpu\n      restartPolicy: NotRequired   # CPU resize without restart\n    - resourceName: memory\n      restartPolicy: RestartContainer  # Memory decrease needs restart\n```\n\nThe default if unset is `RestartContainer`\n\nfor all resources. Set `NotRequired`\n\nfor CPU (safe in both directions) and leave memory as `RestartContainer`\n\n. Most container runtimes as of mid-2025 require a restart for memory resize down; memory resize up is live.\n\n### PodDisruptionBudgets\n\nWhen in-place resize is not available (clusters below 1.33, or memory downsizing), PodDisruptionBudgets (PDBs) limit how many pods VPA or Cast AI can evict simultaneously. A PDB with `maxUnavailable: 1`\n\nallows at most one replica to be taken down during the resize cycle while keeping the rest available.\n\n```\napiVersion: policy/v1\nkind: PodDisruptionBudget\nmetadata:\n  name: my-service-pdb\n  namespace: production\nspec:\n  maxUnavailable: 1\n  selector:\n    matchLabels:\n      app: my-service\n```\n\nAvoid `maxUnavailable: 0`\n\non pods subject to rightsizing. That configuration prevents any voluntary disruption, blocking VPA in Recreate mode and in-place restarts from completing.\n\nSet PDBs before enabling VPA Recreate mode or Cast AI Immediate mode on any service with an SLO. The combination of PDB plus rolling eviction gives you safe restart-based rightsizing on any Kubernetes version.\n\n## Governing Requests\n\nRightsizing individual workloads is necessary but not sufficient. Without namespace-level guardrails, a single misconfigured deployment can consume all cluster resources. Use ResourceQuotas and LimitRanges to enforce governance.\n\n### ResourceQuotas\n\nResourceQuotas cap the total CPU and memory that all pods in a namespace can request and limit. They prevent any single team or workload from crowding out others.\n\n```\napiVersion: v1\nkind: ResourceQuota\nmetadata:\n  name: team-quota\n  namespace: team-a\nspec:\n  hard:\n    requests.cpu: \"10\"\n    requests.memory: 20Gi\n    limits.cpu: \"20\"\n    limits.memory: 40Gi\n```\n\nOnce a ResourceQuota exists in a namespace, every pod must declare requests and limits. Pods without resource declarations fail admission. This forces teams to think about sizing at deploy time. Cast AI Workload Autoscaler is ResourceQuota-aware from v0.82.0 and auto-adjusts recommendations to stay within quota constraints.\n\n### LimitRanges\n\nLimitRanges set default requests and limits for containers that do not specify them, and enforce min/max bounds. They complement ResourceQuotas by operating at the container level rather than the namespace aggregate level.\n\n```\napiVersion: v1\nkind: LimitRange\nmetadata:\n  name: default-limits\n  namespace: team-a\nspec:\n  limits:\n  - type: Container\n    default:\n      cpu: \"500m\"\n      memory: \"512Mi\"\n    defaultRequest:\n      cpu: \"100m\"\n      memory: \"128Mi\"\n    max:\n      cpu: \"4\"\n      memory: \"8Gi\"\n```\n\nWithout a LimitRange, containers that omit resource requests will have no requests set. In namespaces with ResourceQuotas, this makes them unschedulable because quota enforcement requires explicit requests. A LimitRange solves this by injecting `defaultRequest`\n\nvalues automatically. Cast AI Workload Autoscaler is LimitRange-aware and auto-adjusts to comply with namespace constraints.\n\nFor teams on AWS using Karpenter: Karpenter’s NodePool resource constraints act as a cluster-level ceiling on what pods can request. Ensure your NodePool resource limits are set above your namespace ResourceQuotas, otherwise pod scheduling will fail when quota-allowed requests exceed NodePool limits.\n\n## GPU Rightsizing and Fractional GPUs\n\nGPU waste is the most expensive rightsizing problem. The Cast AI 2026 report found average GPU utilization at 5%. A cluster full of A100s running at 5% is not an AI cluster. It is a $10,000-per-node cost center that happens to run some models.\n\nThree approaches share a single GPU across multiple workloads:\n\n**Time-slicing:** The GPU switches between workloads temporally. No memory isolation. Simple to configure. Best for inference workloads that do not need dedicated VRAM.**MIG (Multi-Instance GPU):** Hardware partitioning available on A100 and H100. Provides memory and fault isolation. Each MIG instance looks like a separate GPU to the container. Best for production inference with strict isolation requirements.**NVIDIA MPS (Multi-Process Service):** Process-level sharing for CUDA workloads. Best for training jobs that need to share compute without full isolation overhead.\n\nALLEN Digital moved 7 models from SageMaker to Kubernetes using GPU time-slicing plus Spot instances ([case study](https://cast.ai/case-studies/allen-digital/)). Time-slicing alone delivered 20% savings. Node consolidation added another 30-40%. Total savings exceeded 70%.\n\n## How Much You Can Save\n\nThe 2026 Cast AI data puts CPU overprovisioning at 69% across tens of thousands of clusters. That is the addressable savings pool from CPU rightsizing alone, before touching node autoscaling or Spot usage. Memory overprovisioning at 79% adds to this, though memory savings flow through differently depending on whether you run spot-heavy or reserved fleets.\n\nIn practice, savings depend on how overprovisioned you start. The Bud case study saw up to 93% improvement in CPU and memory utilization via Cast AI automated rightsizing. The OOM cluster in the report cut provisioned CPUs by approximately half while eliminating nearly all kill events.\n\nA useful frame: rightsizing reduces the size of the bin each pod occupies. Node autoscaling then packs more workloads per node and removes underutilized nodes. The two work together. Rightsizing first makes every subsequent autoscaling decision more efficient.\n\nFor a full breakdown of the cost optimization levers and how rightsizing fits into the broader picture, see [Kubernetes Cost Optimization](https://cast.ai/blog/kubernetes-cost-optimization/).\n\n## Conclusion\n\nRightsizing is not a one-time configuration exercise. Clusters change, teams ship new services, and load patterns shift. The clusters that stay optimized are the ones that measure continuously and apply changes automatically. If you manage multiple clusters or frequently deploy new services, Cast AI’s Workload Autoscaler handles the observe-set-verify-repeat loop automatically, recalculating every 30 minutes, reacting immediately to OOM events, and applying changes without disruption. Connect your cluster at [Cast AI](https://cast.ai) to see current recommendations.\n\n**What is Kubernetes rightsizing?** Kubernetes rightsizing is the process of adjusting pod CPU and memory requests and limits to match actual workload usage. Oversized requests waste money because Kubernetes reserves node capacity based on requests, not real consumption. Undersized requests cause throttling (CPU) or OOM kills (memory). Rightsizing closes that gap safely.\n\n**How do I avoid OOM kills?** Set memory requests at p99 or peak observed usage, and set memory limits equal to or slightly above the request. Monitor with `kubectl get events --field-selector reason=OOMKilling`\n\n. Automated tools like Cast AI detect OOMKill events and add memory overhead automatically before the next recommendation cycle, preventing recurrence. For JVM workloads, set -Xmx to 75% of the container memory limit to leave headroom for metaspace and off-heap buffers.\n\n**VPA vs automated rightsizing: which is better?** VPA in Recreate mode applies recommendations but disrupts pods. It retains only 8 days of history and conflicts with HPA when both run on CPU utilization metrics. Automated platforms like Cast AI update every 30 minutes, support in-place resize (no eviction), detect OOM events automatically, and handle LimitRange and ResourceQuota constraints. For production clusters, automated rightsizing reduces ops burden significantly.\n\n**How much can rightsizing save?** The Cast AI 2026 State of Kubernetes Optimization Report found 69% CPU overprovisioning and 79% memory overprovisioning across tens of thousands of production clusters. In practice, savings depend on your starting point. The Bud case study achieved up to 93% improvement in CPU and memory utilization. A reasonable expectation for clusters with no previous optimization is 40-70% reduction in compute costs.\n\n**What tools are available for Kubernetes rightsizing?** The main options are: manual (kubectl top + PromQL), Goldilocks (VPA-based dashboard, read-only), KRR by Robusta (Prometheus-native CLI, longer history windows), VPA (automated apply with multiple modes), and Cast AI Workload Autoscaler (continuous, OOM-aware, in-place resize, quota-aware). Each option trades off accuracy, ops burden, and disruption risk differently.\n\n**How do I rightsize without downtime?** On Kubernetes 1.33+, use in-place pod resizing with the InPlacePodVerticalScaling feature gate (beta in 1.33, GA and default-on in 1.35). CPU and memory resize up happen without a container restart. On older versions, combine PodDisruptionBudgets with rolling eviction to limit simultaneous pod restarts to a safe count. Cast AI supports both approaches and selects the least disruptive method automatically.\n\n**What is GPU rightsizing?** GPU rightsizing matches GPU allocation to actual model utilization. Average GPU utilization in production Kubernetes clusters is 5% (Cast AI 2026). Sharing strategies include time-slicing (temporal sharing, no memory isolation), MIG (hardware partitioning on A100/H100, full isolation), and NVIDIA MPS (process-level sharing for CUDA). Fractional GPU allocation can cut GPU costs by 70% or more without changing model code.", "url": "https://wpnews.pro/news/kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization", "canonical_source": "https://cast.ai/blog/automated-workload-rightsizing-precisionpack/", "published_at": "2026-07-03 11:15:52+00:00", "updated_at": "2026-07-07 00:49:56.679337+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "developer-tools"], "entities": ["Kubernetes", "Cast AI", "AWS", "GCP", "Azure", "NVIDIA"], "alternates": {"html": "https://wpnews.pro/news/kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization", "markdown": "https://wpnews.pro/news/kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization.md", "text": "https://wpnews.pro/news/kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization.txt", "jsonld": "https://wpnews.pro/news/kubernetes-rightsizing-a-practical-workflow-for-cpu-and-memory-optimization.jsonld"}}