# Idle Kubernetes Resources: How to Find and Eliminate Them at Scale > Source: > Published: 2026-11-11 13:57:00+00:00 Idle Kubernetes resources are capacity you paid for and never used. That means the difference between what workloads request and what they actually consume. *Last updated: November 11, 2026* The gap stays hidden because the scheduler reserves against requests, not real usage. A cluster can look full and still run at a fraction of its capacity. At the same time, a node dashboard can read as healthy while most of the reserved CPU sits untouched. To clear it, measure first. Compare requested CPU and memory to P95 actual usage per workload. Then scan for orphaned PersistentVolumeClaims (PVCs), load balancers, and forgotten namespaces. Eliminate what you find in a fixed order: rightsize requests, consolidate onto fewer nodes, delete what is abandoned. The Cast AI 2026 State of Kubernetes Optimization Report covers tens of thousands of clusters. Average CPU utilization across that fleet is 8%. CPU overprovisioning has reached 69%. Most of your idle spend is a configuration problem, and configuration can be fixed this sprint. ## Key Takeaways - Idle equals requested minus used. The scheduler reserves against requests, so idle capacity is invisible to it. A cluster can sit at 8% utilization and still refuse new pods. - Report baseline: average CPU utilization is 8% and memory 20%. CPU overprovisioning has climbed to 69% and memory overprovisioning to 79% (Cast AI 2026 State of Kubernetes Optimization Report, tens of thousands of clusters, measured before optimization). - Five types recur: over-requested workloads, idle pods and replicas, underpacked nodes, orphaned PVCs and load balancers, and forgotten namespaces. - Apply the P95 method: compare requested CPU and memory to P95 actual usage per workload, then rank by the ratio of used to requested. - Sequence matters: rightsize, then consolidate, then delete. Skip that order and you destroy the evidence you need. - Rightsizing is also a reliability win. On one cluster it cut OOM kills from 40 to 50 per interval down to near zero. Organizations applying automated rightsizing cut their provisioned CPU footprint by about 50%, per the same report. ## What counts as an idle Kubernetes resource Start with arithmetic. A pod requesting 2 CPUs and consuming 200 millicores holds 1.8 CPUs of unused capacity. To the scheduler it looks ten times its real size. Most node dashboards and cloud bills report provisioned capacity, not consumption, so neither shows the gap by default. Some FinOps tools surface a request-versus-usage delta once you configure labels and allocation rules, but they stop at the cluster total and rarely name the container to change. Three terms get used as if interchangeable, and the confusion costs engineering time. Overprovisioned means a running workload requests far more than it uses, which is padding on a live service. Unused or orphaned means capacity with no workload behind it, such as a PVC whose pod was deleted. Every paid-for capacity doing no useful work is idle, including nodes nobody packed. [Kubernetes rightsizing](https://cast.ai/blog/automated-workload-rightsizing-precisionpack/) handles the overprovisioned case. The orphaned and underpacked cases need a scan. ## The five types of idle Kubernetes resources Nearly every wasted dollar in a cluster falls into one of five buckets. Each has a distinct detection signal, which matters because the fix for one does not touch the others. 1. **Over-requested workloads.** Detection: requested CPU or memory far above P95 actual usage for the same container. 2. **Idle pods and replicas.** Detection: replicas serving near-zero traffic, often a Deployment pinned at a non-zero count or an HPA with a floor above zero. 3. **Underpacked nodes.** Detection: node allocatable far above the sum of pod usage on that node. 4. **Orphaned PVCs and load balancers.** Detection: a bound volume or cloud load balancer with no owning pod or Service. 5. **Forgotten namespaces.** Detection: no recent deployments and near-zero usage across a representative window. The table is the short reference for all five. Pay attention to the right column. Detection is rarely where cleanup goes wrong. Order of operations is. | Idle type | Detect | Eliminate | Safety note | |---|---|---|---| | Over-requested workloads | Requested far above P95 actual | Rightsize to P95 plus headroom | Keep spike headroom | | Idle pods / replicas | Replicas serving near-zero traffic | Scale down; set HPA min | Do not scale stateful blindly | | Underpacked nodes | Allocatable far above pod usage | Bin-pack onto fewer nodes | Respect PDBs and anti-affinity | | Orphaned PVCs and LBs | Bound, no owner pod | Delete after verifying owner | Snapshot PVCs first | | Forgotten namespaces | No recent activity | Archive after sign-off | Check for cron jobs first | ## How to find idle Kubernetes resources Measure before you remove anything. One cluster average tells you little, because Kubernetes waste concentrates. In a shared cluster, one namespace running a fat legacy service can hold most of the padding while every other team sits near the median. You want a ranked list by workload, not a headline number. The largest [Kubernetes cost drivers](https://cast.ai/blog/kubernetes-cost-drivers/) on most bills, over-requested pods and idle nodes, both surface there. Ranking is where cleanup starts. ### The P95 method: requested vs actual usage Compare requested CPU and memory to P95 actual consumption per workload, over seven to fourteen days. A one-week window misses monthly batch jobs, and a single day misses anything on a cron. The gap between request and P95 is your padding estimate, and it usually runs larger than engineers expect. The ratio that ranks workloads is used divided by requested, expressed as a percentage. Take a container at 200 millicores of P95 usage against a 1,000 millicore request. That sits at 20%. That is one signal among many, and it needs a second reference point. Compare each workload against its own history before you change a request. A batch job that peaks once a day still needs its peak covered. The following recording rules compare per-container P95 usage with positive current requests. They collapse duplicate scrape labels within one cluster and exclude the infrastructure container. Containers without positive requests need a separate audit. Keep these series to compare before and after changes. ``` # Single-cluster example; retain a cluster label for multi-cluster queries. groups: - name: container-utilization interval: 5m rules: - record: container:cpu_p95_ratio expr: | quantile_over_time(0.95, (max by (namespace, pod, container) ( rate(container_cpu_usage_seconds_total{container!="",container!="POD",namespace!="kube-system"}[5m]) ))[7d:5m]) / on(namespace, pod, container) (max by (namespace, pod, container) ( kube_pod_container_resource_requests{resource="cpu",namespace!="kube-system"} ) > 0) - record: container:memory_p95_ratio expr: | quantile_over_time(0.95, (max by (namespace, pod, container) ( container_memory_working_set_bytes{container!="",container!="POD",namespace!="kube-system"} ))[7d:5m]) / on(namespace, pod, container) (max by (namespace, pod, container) ( kube_pod_container_resource_requests{resource="memory",namespace!="kube-system"} ) > 0) ``` Two caveats apply at fleet scale. A seven-day subquery stepped at five minutes evaluates about 2,016 samples per series, and `rate()` inside it compounds the cost. Expect slow queries across a few thousand workloads and resource limits on smaller Prometheus deployments, so split by namespace or raise the step on long-lived workloads. Results also inherit your scrape interval, so compare workloads scraped at the same cadence. Use `container_memory_working_set_bytes` as one memory-sizing signal, alongside limits, RSS, cache behavior, and OOM history. It is not a direct OOM threshold. The rules above retain pod and container labels; rollouts create new series. For stable workload trends, map pods to their Deployment or StatefulSet owners before aggregating, and use consistent cluster labels when querying multiple clusters. ### Prometheus queries and kubectl checks Without recording rules, a few commands give you a first pass in a couple of minutes. Start with the pods themselves. ``` # Actual CPU and memory per container, next to the request kubectl top pods -n payments --containers kubectl get pods -n payments -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.containers[*]}{.name}{"="}{.resources.requests.cpu}{"/"}{.resources.requests.memory}{" "}{end}{"\n"}{end}' ``` Node-level checks confirm underpacking. The gap between requested and used capacity on each node is where consolidation gets its headroom. ``` kubectl describe node | grep -A5 'Allocated resources' kubectl top nodes # Single-cluster namespace triage: sum of per-container P95 usage / requests. sum by (namespace) ( quantile_over_time(0.95, (max by (namespace, pod, container) ( rate(container_cpu_usage_seconds_total{container!="",container!="POD"}[5m]) ))[7d:5m]) and on(namespace, pod, container) (max by (namespace, pod, container) ( kube_pod_container_resource_requests{resource="cpu"} ) > 0) ) / (sum by (namespace) ( max by (namespace, pod, container) ( kube_pod_container_resource_requests{resource="cpu"} ) ) > 0) ``` A namespace ratio below 0.25 can be a useful initial review threshold; below 0.10 warrants closer inspection. These are illustrative triage cutoffs, not proof that a workload is abandoned. Compare against scheduled peaks and rank candidates by estimated savings as well as utilization. ### Scan for orphaned PVCs, load balancers, and namespaces Orphans never appear in utilization data, since no workload runs to generate usage. Cross-reference resources against their owners instead. For PVCs, a Bound claim with no active pod referencing it is a cleanup candidate, not proof that its data is disposable. Dynamically provisioned volumes inherit their StorageClass reclaim policy, which defaults to `Delete`; volumes configured with `Retain` keep their backing storage after claim deletion. Check the actual PV policy before acting. A retained PV can enter `Released` while its cloud disk continues billing. See the [Kubernetes volume lifecycle documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). Also check scaled-to-zero StatefulSets, suspended Jobs, and CronJob templates for future use. ``` # Bound PVCs not referenced by any non-terminal pod (review candidates only) kubectl get pvc -A -o json | jq -r '.items[] | select(.status.phase=="Bound") | .metadata | "\(.namespace)/\(.name)"' | sort > /tmp/pvcs.txt kubectl get pods -A -o json | jq -r '.items[] | select(.status.phase!="Succeeded" and .status.phase!="Failed") | .metadata.namespace as $ns | .spec.volumes[]? | select(.persistentVolumeClaim) | "\($ns)/\(.persistentVolumeClaim.claimName)"' | sort -u > /tmp/mounted.txt comm -23 /tmp/pvcs.txt /tmp/mounted.txt # Released PVs left behind by a Retain policy kubectl get pv | grep Released ``` Per-object cost is modest, but it compounds. A 100 GB gp3 EBS volume runs about $8 per month, and a GKE 100 GB SSD persistent disk about $17, so twenty orphaned PVCs averaging 50 GB cost roughly $80 to $170 monthly. Load balancers cost more, so thirty unused NLBs at $16 each run near $500. For load balancers, no ready endpoints plus no traffic is a signal to investigate, not proof of abandonment. A provisioned cloud load balancer can keep billing while its Service has no healthy backends. List Services, filter by `spec.type` in jq, and inspect the provisioning state and application ownership before cleanup. ``` # Every LoadBalancer Service with its address, or PENDING kubectl get svc -A -o json | jq -r '.items[] | select(.spec.type=="LoadBalancer") | "\(.metadata.namespace) \(.metadata.name) \(.status.loadBalancer.ingress[0].ip // .status.loadBalancer.ingress[0].hostname // "PENDING")"' | sort # Ready endpoints per LoadBalancer Service, summed across every slice kubectl get endpointslices -A -o json | jq -r '.items[] | select(.metadata.labels["kubernetes.io/service-name"] != null) | [(.metadata.namespace), (.metadata.labels["kubernetes.io/service-name"]), ((.endpoints // []) | map(select(.conditions.ready == true)) | length)] | @tsv' | awk -F'\t' '{sum[$1"/"$2]+=$3} END {for (k in sum) print k, sum[k]}' | sort ``` Compare the Service addresses with endpoint counts. An address with no ready endpoints can indicate an unused resource, a temporary outage, or a service waiting for deployment. PENDING means provisioning is incomplete; verify whether a cloud resource already exists. Selector-less Services may use manually managed EndpointSlices, and `publishNotReadyAddresses` changes readiness semantics. Check traffic history and ownership before deciding a load balancer is orphaned. One aggregation trap sits in that jq. Kubernetes uses a default target of up to 100 endpoints per EndpointSlice, but the limit is configurable and a Service can have multiple slices below that total. Counting ready endpoints per slice without summing every slice carrying `kubernetes.io/service-name` for that Service undercounts the total, and a busy Service looks idle. Aggregate across all slices per namespace and service name first. Namespaces need a different scan. Sort by creation time, then look for entries with no deployments and flat usage. Check cron jobs first, since a nightly batch job in staging looks dead for 23 hours. Get owner sign-off before archiving, because the abandoned-looking namespace is sometimes the DR environment. ``` kubectl get ns --sort-by=.metadata.creationTimestamp kubectl get deploy,statefulset,daemonset,job,cronjob -n # Review current CPU and memory alongside historical monitoring. kubectl top pods -n ``` ## How to eliminate idle resources (in order) The order is rightsize, consolidate, then delete, and it is not arbitrary. Deleting before rightsizing hides the real signal. You lose the artifact that proves which workload was over-requesting, and the next deployment re-creates it from the same chart. Consolidation only reclaims what rightsizing freed. A node that looks underpacked on usage is often correctly packed on requests. Skip step one and your bin-packer will decide the cluster is full. ### 1. Rightsize requests first Use P95 CPU usage plus measured headroom as a starting point, then validate under peak traffic. CPU limits can cause throttling; choose them from latency tests rather than a universal multiplier. For memory, account for P99 usage, known batch peaks, and OOM history. Equal memory requests and limits alone do not give Guaranteed QoS: with container-level resources, every container needs positive, matching requests and limits for both CPU and memory. Guaranteed pods can still be evicted under node pressure or killed when they hit a memory limit. Review [Kubernetes requests and limits](https://cast.ai/blog/kubernetes-requests-and-limits/) before changing either setting. Here is an illustrative capacity calculation, not a deployment recommendation. A frontend with 12 replicas requesting 1000m CPU and 2Gi memory reserves 12 vCPU and 24Gi. If observed CPU P95 is 250m, a 320m request gives 28% headroom. A 1Gi memory request is about 14% above 900Mi, but memory peaks and limits still need separate validation. The revised total is 3.84 vCPU and 12Gi. Actual node count depends on allocatable memory and CPU, system overhead, burst demand, and replica spreading; do not collapse a highly available service onto one node. ``` # Illustrative request comparison, not an apply-ready manifest. # Before After # Replicas 12 12 # CPU request per pod 1000m 320m # Memory request per pod 2Gi 1Gi # Total requested CPU 12 vCPU 3.84 vCPU # Total requested memory 24Gi 12Gi # Validate peaks, allocatable capacity, and replica spreading before consolidation. ``` Check throttling and application latency during rollout and again after a representative traffic cycle. A high fraction of throttled CFS periods is an investigation signal, not a universal latency threshold. The PromQL sits in the [rightsizing workflow](https://cast.ai/blog/automated-workload-rightsizing-precisionpack/). Rightsizing usually gets pitched as a cost cut. Yet Cast AI’s Workload Autoscaler also raises memory limits on workloads that were starving. The [2026 State of Kubernetes Optimization Report](https://cast.ai/reports/state-of-kubernetes-optimization/) tracks one representative cluster. It ran 40 to 50 OOM kills per measurement interval, with spikes above 80. After automated rightsizing with OOM-detection overhead, kills fell to near zero while provisioned CPUs dropped by roughly half. Mercedes-Benz.io hit a related problem one layer down. Un-rightsized workloads sat next to outdated instance types, so cost kept climbing even as they tuned. The fix combined automated instance selection, node autoscaling, and periodic rebalancing with bin-packing, all Cast AI components. Workload Autoscaler moved requests up and down with usage, and both cost and operational overhead fell. The [case study](https://cast.ai/case-studies/mercedes-benz-io/) publishes a utilization curve rather than a headline percentage. Across the fleet, organizations applying automated rightsizing cut their provisioned CPU footprint by about 50%, per the same report. That is the headroom the P95 pass in step one uncovers. ### 2. Consolidate with bin-packing Once requests are accurate, pack workloads onto fewer, right-sized nodes. Bin-packing decides placement at schedule time, while consolidation finds underutilized nodes later, migrates pods, and deletes the empties. You need both, since fixing requests only changes where pods land. Before consolidation, check replica counts, PodDisruptionBudgets, graceful termination, and topology constraints together. PDBs limit voluntary disruptions; they do not drain application requests or protect against every failure. Anti-affinity and topology spread keep replicas separated when the workload requires it. A budget that permits no disruption can block consolidation, so test the policy against the availability target. Audit them before you enable anything. ``` # All PDBs kubectl get pdb -A ``` For [Karpenter consolidation](https://cast.ai/blog/karpenter-consolidation/), configure `consolidationPolicy: WhenEmptyOrUnderutilized` and a NodePool disruption budget. The `nodes: "10%"` budget limits voluntary disruptions, with rounding and allowances for deleting or NotReady nodes; it does not cap every interruption or forced expiration. Merge this fragment into an existing NodePool, preserving its template and node class configuration. ``` apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: workers spec: disruption: consolidationPolicy: WhenEmptyOrUnderutilized consolidateAfter: 5m budgets: - nodes: "10%" ``` Draining by hand skips that guardrail. `kubectl drain` is disruptive by default and honors PDBs only through the eviction API, so check `kubectl get pdb -A` first. Karpenter’s consolidation handles node-level reclamation, but it reads the same requests your pods declare, and padded requests leave it nothing to consolidate. The [bin-packing guide](https://cast.ai/blog/kubernetes-bin-packing/) shows how to configure the reclamation loop. ### 3. Delete orphaned resources safely Delete last, and deliberately. Label the candidate cleanup, then wait a few business days for objections. After that, remove the parent workload, the Service, its ConfigMaps and Secrets, then the PVC. Storage goes last, since it holds the data. Snapshots are not optional. Take a VolumeSnapshot before deleting a PVC. After that, delete the claim, check for Released PVs and remove them, and confirm the cloud disk is gone. Orphans survive at two layers independently. A claim removed from the API can leave an EBS volume billing in your AWS account. After deleting a LoadBalancer Service, verify at the provider too: `aws elbv2 describe-load-balancers`, `gcloud compute forwarding-rules list`, or `az network public-ip list`. If the cloud controller lost track of the resource, the Service object disappears and the LB keeps running. Reclaim policy is a data-retention decision, not a cleanup setting. It lives at the PV or StorageClass level, and each class needs owner sign-off. For a scratch or cache class, `Delete` is defensible. For anything holding state, `Retain` is safer even when it leaves Released PVs behind, because a leaked disk is recoverable and a deleted one is not. Write down the retention window per class. One number sets the stakes: industry cloud cost data puts the majority of container spend on capacity that is provisioned but unused, as summarized in Cast AI’s [Kubernetes cost optimization guide](https://cast.ai/blog/kubernetes-cost-optimization/). Prevention beats repeated cleanup. Cap storage with a ResourceQuota on `requests.storage`, and add a LimitRange default request in every namespace so new workloads arrive with requests set. ## Why this cannot be done manually at scale Idle drifts back, and toward more. Requests inflate after the next incident, since the fastest fix at 2 a.m. is a bigger request. New services arrive from Helm charts with conservative defaults, and P95 shifts as workloads change, invalidating sizing from a quarter ago. A one-off audit buys a week of accurate numbers and a stale spreadsheet after that. What holds idle down is a continuous loop: rightsize on fresh percentiles, bin-pack placements, run an evictor that reclaims freed nodes. [Cast AI’s automation](https://cast.ai/kubernetes-cost-optimization/) can automate that loop through workload rightsizing, node selection, and evictor-based rebalancing, according to the features and policies enabled for the cluster. To build it yourself, the [Kubernetes cost optimization checklist](https://cast.ai/blog/kubernetes-cost-optimization-checklist/) walks through each step. To see Cast AI run this loop on your clusters, [book a demo](https://cast.ai/book-a-demo/). ## Frequently asked questions ### How do I find unused PVCs in Kubernetes? List PersistentVolumeClaims and cross-reference them against mounting pods. A Bound claim with no pod mounting it is a candidate for removal. Check unscheduled Jobs and CronJob templates first, then snapshot the claim before deleting. ### How do I detect idle namespaces? List the workloads that actually live there with `kubectl get deploy,statefulset,daemonset,job,cronjob -n `, then pull a seven day usage sample for each namespace you suspect. A namespace with deployments present but near-zero CPU and memory over that window is a candidate for consolidation, while one with no workloads at all is a candidate for deletion once you confirm the owners. ### Why does my cluster look full at 8% utilization? Because the scheduler reserves capacity against requests, not measured use. Padded requests fill the cluster on paper, allocatable capacity runs out, and real CPU usage stays near 8 percent. ### Does the Kubernetes HPA fix idle resources? The HPA scales replica counts in and out as demand moves, which reclaims capacity tied to traffic swings. It never shrinks the requests on a pod that is running, so a service sitting at ten percent of its declared request stays padded at every replica count; pair the HPA with rightsizing to fix both layers. ### What is a safe deletion policy for idle resources? Confirm ownership and retention requirements, verify a usable backup or snapshot, and check the reclaim policy before deleting storage. After cleanup, confirm that the intended cloud resources are gone. Before consolidation, validate replicas, PDBs, topology constraints, and graceful shutdown. ### How do I do chargeback for idle resources? Allocate cost by namespace or label, then show requested versus used side by side using a [Kubernetes chargeback and showback](https://cast.ai/blog/kubernetes-chargeback-showback/) model. Each team sees its own idle rather than a cluster-wide average. ### What is the difference between idle, unused, and overprovisioned resources? Idle covers all three, and each fix is different: rightsize the padding, scan for the orphan, pack the nodes.