Kubernetes Requests and Limits: How to Right-Size Pods Without Breaking Reliability

Kubernetes requests and limits determine pod scheduling and runtime resource enforcement, but the Cast AI 2026 State of Kubernetes Optimization Report finds average CPU utilization across production clusters is just 8%, measured directly from tens of thousands of clusters. The gap between requested and actual usage drives both cost and stability risk, and the guide explains how to derive correct values from real data and enforce them at scale.

Kubernetes requests and limits are the two resource controls that determine where pods get scheduled, how much CPU and memory they can consume, and what happens when they exceed those boundaries. According to the Cast AI 2026 State of Kubernetes Optimization Report https://cast.ai/reports/state-of-kubernetes-optimization/ , average CPU utilization across production clusters sits at just 8%. That 8% figure comes from direct measurement across tens of thousands of production clusters; not survey estimates. That gap between what teams request and what workloads actually use costs real money and creates real stability risk. This guide walks through the mechanics, the failure modes, how to derive correct values from real data, and how to enforce them at scale. Requests vs. Limits: How Kubernetes Schedules Pods What Each Setting Actually Does A request is the amount of CPU or memory Kubernetes guarantees to a container. The kube-scheduler uses requests, and only requests, to decide which node can host a pod. If a node’s allocatable CPU minus its total scheduled requests is less than the incoming pod’s CPU request, the scheduler skips that node entirely. A limit is the maximum a container can consume. Limits are invisible to the scheduler. They only matter after the pod is running. Once a container hits its CPU limit, the kernel throttles it. Once it hits its memory limit, the kernel kills it. The table below captures the functional difference between the two settings. | Property | Requests | Limits | |---|---|---| | What it does | Guaranteed minimum for scheduling and resource reservation | Maximum the container can consume at runtime | | Used for scheduling | Yes — scheduler places pods based on requests | No — invisible to kube-scheduler | | CPU enforcement mechanism | cpu.shares cgroup v1 / cpu.weight cgroup v2 : proportional share under contention | cpu.cfs quota us / cpu.cfs period us cgroup v1 / cpu.max cgroup v2 : hard cap per CFS period | | Memory enforcement mechanism | Soft guarantee via QoS eviction priority | Hard cap: container is OOM killed on breach | | Failure mode if set too high | Pods stuck in Pending; wasted capacity; unnecessary node scale-out | N/A high limits are permissive, not dangerous | | Failure mode if set too low | Pods scheduled incorrectly; resource starvation under contention | CPU throttling P99 latency spikes or OOMKilled exit code 137 | QoS Classes: Eviction Priority Under Pressure Kubernetes assigns every pod a Quality of Service QoS class based on how requests and limits are configured. This class determines which pods the kubelet evicts first when a node runs low on memory. | QoS Class | Condition | Eviction Priority | |---|---|---| | Guaranteed | Every container in the pod has memory limit = memory request AND cpu limit = cpu request | Last — evicted only under critical node pressure | | Burstable | At least one container has requests or limits set, but the pod does not meet Guaranteed criteria | Middle — evicted after BestEffort pods are gone | | BestEffort | No container in the pod has any requests or limits set | First — evicted immediately under any memory pressure | Guaranteed pods have an additional benefit: they qualify for exclusive CPU allocation via the static CPU management policy. For latency-sensitive workloads, this eliminates the CPU sharing overhead that causes jitter under load. A note on eviction precision: the kubelet’s QoS-based eviction ordering applies when it detects memory pressure and proactively evicts pods. However, the Linux OOM killer which acts when the kernel itself runs out of memory uses oomScoreAdj scores that only partially align with QoS class. Under extreme kernel-level memory pressure, Kubernetes can kill a Guaranteed pod before a Burstable pod that uses less memory. The OOM killer targets the process with the highest combined score, which accounts for both the adjustment and actual RAM consumption. BestEffort pods are always killed first. How Over-Setting Requests Wastes Resources and Money The 2026 Cast AI State of Kubernetes Optimization Report measured 69% of requested CPU going unused across production clusters. That’s up from 40% the prior year the waste is getting worse, not better . Memory overprovisioning sits at 79%. These are not outliers from small teams. These are fleet-wide averages across organizations running Kubernetes at scale. Why Teams Overprovision The root cause is straightforward: overprovisioning feels safe. Requesting more CPU prevents throttling. Requesting more memory prevents OOM kills. However, the cost compounds silently across every pod, every namespace, and every cluster. Over-setting requests hurts in two concrete ways. First, the cluster autoscaler sees nodes as full before they actually are, so new nodes spin up based on scheduled requests, not actual usage. Second, the scheduler can starve other pods when inflated requests make nodes appear full on paper. The Scale Problem Consider a workload that uses 500m CPU at p95 but has a request of 2 cores. That pod runs 4x overprovisioned. At 500 replicas across a cluster, that is 750 wasted cores. Those cloud costs add a meaningful line item to the monthly bill before a single engineer does any optimization work. At 500 nodes, the compounding is more severe. Overprovisioned requests cause the cluster autoscaler to hold 20 to 30% more nodes than the workload needs. Because autoscaler decisions are request-driven, not usage-driven, accurate requests are the prerequisite for efficient autoscaling. The autoscaling guides for HPA https://cast.ai/blog/what-is-kubernetes-hpa-and-how-can-it-help-you-save-on-the-cloud/ and cluster-level autoscaling https://cast.ai/blog/guide-to-kubernetes-autoscaling-for-cloud-cost-optimization/ both assume requests that reflect real usage. Failure Modes: OOMKilled Exit Code 137 and CPU Throttling Under-setting requests and limits produces two distinct failure modes. Both are preventable with the right values. Both are diagnosable with the right tooling. OOMKilled: Memory Limit Breached When a container exceeds its memory limit, the Linux OOM killer sends SIGKILL signal 9 to the process. The exit code is 128 + 9 = 137. Kubernetes reports the container status as OOMKilled . Confirm it with: kubectl describe pod <pod-name -n <namespace Look for OOMKilled: true in the container status section. Two root causes typically drive the problem: the workload legitimately peaks beyond its memory limit, or the container has a memory leak. Distinguish them by charting memory over time. A workload with a leak shows continuously growing memory consumption. A workload hitting a hard ceiling shows usage that spikes to a fixed value and then resets after the kill and restart cycle. For a complete diagnosis and remediation guide, see the Cast AI reference on OOMKilled exit code 137 https://cast.ai/blog/oomkilled-exit-code-137/ . JVM Workloads Require Explicit Heap Sizing Java applications without explicit heap bounds attempt to allocate a percentage of total available memory. In a container, that means the node’s memory — not the container limit. This reliably causes OOMKilled events even when the Kubernetes memory limit is correctly set. Always set JVM heap bounds explicitly: -XX:MaxRAMPercentage=75.0 : limits heap to 75% of the container memory limit, leaving room for JVM overhead, thread stacks, and native memory allocation- Or explicit -Xmx and -Xms values derived from the container’s memory limit, not the node’s memory Without these flags, a Java process grows past the container limit and triggers OOMKilled regardless of how accurately the Kubernetes memory limit is configured. Getting the Kubernetes limit right is necessary but not sufficient for JVM workloads. CPU Throttling: The Silent Latency Problem CPU throttling is subtler than OOM kills because the container keeps running. The Linux Completely Fair Scheduler CFS enforces CPU limits via cpu.cfs quota us and cpu.cfs period us in cgroup v1, or cpu.max in cgroup v2. The default accounting period is 100ms. A CPU limit of 500m means the container receives at most 50ms of CPU time per 100ms window. Once the quota for that window is exhausted, the container waits until the next period. This happens regardless of whether other CPUs on the node are idle. The CFS accounting window is fixed, and it does not consider node-wide availability. The result is P99 latency spikes that correlate with traffic bursts rather than infrastructure events. These are difficult to root-cause without specifically monitoring the throttle ratio. Measure throttling with this PromQL expression. The label selectors exclude pause containers and infrastructure sidecars, which would otherwise pollute the results: rate container cpu cfs throttled periods total{container ="", container ="POD"} 5m / rate container cpu cfs periods total{container ="", container ="POD"} 5m If this ratio exceeds 25% for a given container, raise the CPU limit. For latency-sensitive workloads, consider removing the CPU limit entirely and relying on requests for scheduling fairness. Cast AI tracks throttle ratios across the entire fleet and surfaces workloads where CPU limits are causing measurable latency issues, without requiring per-team Prometheus dashboards. Should You Set CPU Limits? The Answer Depends on Your Workload The answer differs by workload type: Latency-sensitive services APIs, streaming, ML inference : Omit CPU limits, or set them at 2–4x the CPU request. The CFS throttling risk outweighs the noisy-neighbor protection benefit. Batch workloads data processing, nightly jobs : Set CPU limits. They help enforce fair-sharing without P99 latency consequences. Multi-tenant namespaces : Always set limits, paired with a LimitRange. Namespace quota enforcement via ResourceQuota requires limits. Read the reasoning below to understand why, and to decide if your specific situation differs. Why CPU Limits Cause Latency Problems CFS throttling causes measurable P99 latency degradation even when the node has spare CPU capacity. Numerator Engineering documented this directly: removing CPU limits from their ML inference pods eliminated throttling entirely and improved tail latency. The underlying issue is that limits set for average load penalize burst traffic through the fixed CFS accounting window, even when the hardware has no contention. When CPU Limits Help In a multi-tenant cluster, CPU limits protect neighbors from noisy processes. Without limits, a single misbehaving container can consume all available CPU on a node and starve every other container. Furthermore, ResourceQuota namespace enforcement requires limits to be declared. If you operate a shared cluster with per-team quotas, you need limits to make those quotas meaningful. In all cases, monitor container cpu cfs throttled periods total / container cpu cfs periods total with the label selectors above. If the ratio climbs above 25%, the current limit is too low for the workload’s burst profile. How to Derive Safe Kubernetes Requests and Limits Values Observe Before You Configure The correct approach is to measure what the workload actually uses, then set values from data. Never guess. Use Prometheus to observe real usage over a meaningful window: two weeks minimum, four weeks for workloads with weekly traffic cycles. For CPU requests, use p95 of the usage rate over a two-week observation window. The label selectors below exclude pause containers and infrastructure sidecars that would inflate the results. The max by clause groups results per container so each service gets an independent recommendation: CPU request baseline: p95 usage per container over 2 weeks result in millicores max by namespace, pod, container quantile over time 0.95, rate container cpu usage seconds total{container ="", container ="POD"} 5m 2w:5m 1000 For memory limits, use p99 of working set bytes. Working set is preferred over RSS because it excludes reclaimable cache pages, which the kernel reclaims under memory pressure. RSS includes them, so it overstates actual memory pressure: Memory limit baseline: p99 working set per container over 2 weeks max by namespace, pod, container quantile over time 0.99, container memory working set bytes{container ="", container ="POD"} 2w:5m Retention requirement: Both subqueries use a 2-week lookback window 2w:5m . Your Prometheus instance must retain at least 14 days of metrics for these queries to return meaningful results. If retention is shorter, reduce the window to match your actual retention period, and note that the percentile estimates will be less representative of weekly traffic cycles. Spiky workloads: For workloads with spike patterns – JVM GC pauses, traffic bursts, batch processing – p99 may not provide sufficient headroom. Consider using max container memory working set bytes{container ="", container ="POD"} over the same window as a more conservative ceiling, then adding 20% headroom above that value. HPA and VPA: Do Not Run Both on CPU Running Horizontal Pod Autoscaler HPA and Vertical Pod Autoscaler VPA simultaneously on the same workload with CPU as the scaling signal causes oscillation. VPA adjusts the CPU request, which changes the utilization denominator that HPA uses for its scaling decisions. Both controllers fight each other in a feedback loop, and neither converges. Three safe patterns for combining the two: HPA on CPU + VPA in Use VPA recommendations as a manual sizing reference without enabling automatic application. HPA controls replica count; you update requests manually from VPA output. Off mode: HPA on custom metrics + VPA in Decouple the scaling signals. HPA scales on queue depth or request rate; VPA adjusts resource sizing independently without touching the CPU utilization denominator. Recreate or InPlaceOrRecreate : VPA alone no HPA : For workloads without horizontal scaling needs, let VPA handle all resource adjustments. Using VPA for Automated Observation The Vertical Pod Autoscaler VPA automates this observation-and-recommendation loop. It watches historical usage and generates sizing recommendations. VPA has four operating modes, each with different disruption profiles: Off: No disruption. Recommendations only – read with kubectl get vpa . Safe to enable immediately on any workload. Initial: No disruption to running pods. Applies recommendations at pod creation or restart only. Use this when you want new pods to start with right-sized values but cannot tolerate mid-session restarts. Recreate: Restarts pods to apply updated recommendations. Use only for workloads that can tolerate brief downtime; pair with a PodDisruptionBudget to control the disruption rate. InPlaceOrRecreate VPA 1.4.0+, Kubernetes 1.33+ : No restart for CPU adjustments. Falls back to Recreate for memory adjustments when in-place resize is not supported. This mode is the right default for teams on recent Kubernetes versions who want continuous rightsizing without pod restarts. VPA Auto mode deprecation: The Auto updateMode is deprecated as of VPA 1.4.0 and now aliases Recreate . Teams with existing VPA manifests using updateMode: Auto should migrate to updateMode: Recreate explicitly. Leaving Auto in place will continue to work, but the behavior is now identical to Recreate and the implicit aliasing may change in a future VPA release. Start with Off mode to collect two weeks of recommendations before switching to any active mode. This avoids applying VPA to workloads that have not yet reached stable traffic patterns. Cast AI Workload Autoscaler uses VPA-style recommendations with built-in support for in-place resize on Kubernetes 1.35+, so teams avoid pod restarts during routine rightsizing cycles without managing VPA manifests directly. A Practical Deployment YAML The following Deployment spec shows correct resource configuration for a latency-sensitive API service. CPU limit is intentionally omitted to avoid CFS throttling. Memory limit is set to prevent OOM kills. apiVersion: apps/v1 kind: Deployment metadata: name: api-server namespace: production spec: replicas: 3 selector: matchLabels: app: api-server template: metadata: labels: app: api-server spec: containers: - name: api-server image: myorg/api-server:v1.4.2 resources: requests: cpu: "250m" p95 of observed usage over 2 weeks memory: "256Mi" baseline working set limits: memory: "512Mi" p99 working set + 25% headroom cpu limit omitted: latency-sensitive service monitor container cpu cfs throttled periods total if throttle ratio 25%, add: cpu: "1000m" Governance: ResourceQuota, LimitRange, and Admission Defaults Individual pod configuration handles the per-workload layer. Governance handles the cluster-wide and namespace-wide layer. Together, they prevent resource sprawl as teams grow and workloads multiply. ResourceQuota: Namespace-Level Hard Caps ResourceQuota sets hard limits on total resource consumption for an entire namespace. When a namespace has an active quota, no team can exceed the combined resource ceiling, regardless of what individual pods request. apiVersion: v1 kind: ResourceQuota metadata: name: production-quota namespace: production spec: hard: requests.cpu: "20" total CPU requests across all pods requests.memory: "40Gi" total memory requests across all pods limits.cpu: "40" total CPU limits across all pods limits.memory: "80Gi" total memory limits across all pods pods: "100" maximum pod count in namespace This quota blocks new pods once the namespace collectively requests more than 20 cores or 40Gi of memory. Existing pods continue running. Only new scheduling requests are rejected. LimitRange: Per-Container Defaults and Constraints LimitRange operates at the container level, not the namespace level. It serves two purposes: injecting default requests and limits for containers that do not specify them, and enforcing per-container minimums and maximums. apiVersion: v1 kind: LimitRange metadata: name: default-limits namespace: production spec: limits: - type: Container default: applied as limits if not specified cpu: "500m" memory: "256Mi" defaultRequest: applied as requests if not specified cpu: "100m" memory: "128Mi" max: container cannot exceed these values cpu: "4" memory: "4Gi" min: container must request at least these values cpu: "50m" memory: "64Mi" A Critical Deployment Order Gotcha There is a failure mode that catches teams by surprise. If you deploy a ResourceQuota without a LimitRange, any pod that omits explicit resource specs will be rejected with HTTP 403. The reason: the quota requires every pod to declare resources so they count against the cap , but without LimitRange defaults, pods with no specs have nothing to satisfy that requirement. Always deploy LimitRange before or alongside ResourceQuota. This ensures that pods without explicit resource specs inherit sensible defaults rather than failing admission. For a thorough governance walkthrough, including patterns for managing quotas across many namespaces, see the Kubernetes ResourceQuota and LimitRange deep-dive https://cast.ai/blog/kubernetes-resource-quotas-limitranges/ . Conclusion Kubernetes requests and limits are not a one-time configuration. They drift as workloads evolve, traffic patterns shift, and application behavior changes. The values that were accurate six months ago may now be wasteful or insufficient. The operational model that holds up at scale: observe with Prometheus over two to four weeks using label-filtered queries that exclude pause containers, set CPU requests at p95, set memory limits at p99 working set with headroom higher for spiky workloads , govern with LimitRange defaults and ResourceQuota caps, and revisit on a regular cadence. For teams managing dozens or hundreds of workloads, this cadence is hard to sustain manually. That is where automated rightsizing https://cast.ai/blog/automated-workload-rightsizing-precisionpack/ with the Cast AI Workload Autoscaler delivers compounding returns: it continuously observes real usage, generates recommendations, and applies adjustments without manual intervention or pod restarts on supported Kubernetes versions. The most impactful change for most teams is setting CPU requests to p95 observed usage and removing CPU limits from latency-sensitive services. Start there. Frequently Asked Questions What is the difference between requests and limits in Kubernetes? A request is the amount of CPU or memory Kubernetes guarantees to a container and uses for scheduling decisions. A limit is the maximum the container can consume at runtime. The scheduler places pods based on requests only; limits are invisible to scheduling. If a container exceeds its CPU limit, the kernel throttles it. If it exceeds its memory limit, the kernel kills it with exit code 137 OOMKilled . What causes OOMKilled in Kubernetes? OOMKilled occurs when a container exceeds its memory limit. The Linux OOM killer sends SIGKILL signal 9 , producing exit code 137. There are two causes: the memory limit is set too low for the workload’s peak usage, or the container has a memory leak. Distinguish them by charting memory over time. A workload with a leak shows continuously growing memory. A workload hitting a fixed ceiling shows memory that spikes to the limit and resets after each restart. For JVM workloads, also verify that explicit heap bounds are set — a Java process without -XX:MaxRAMPercentage or -Xmx will size its heap against node memory, not the container limit. Should I set CPU limits in Kubernetes? It depends on the workload. For latency-sensitive services APIs, real-time inference , omit CPU limits or set them at 2-4x the CPU request to avoid CFS throttling. CFS throttling can spike P99 latency even when the node has idle CPUs, because the accounting window is fixed at 100ms. For batch workloads and multi-tenant namespaces, set CPU limits to enforce fair sharing. Monitor container cpu cfs throttled periods total{container =””,container =”POD”} / container cpu cfs periods total{container =””,container =”POD”}; if this ratio exceeds 25%, raise the CPU limit. How do resource requests affect Kubernetes cost? The kube-scheduler and cluster autoscaler both make decisions based on requests, not actual usage. Overprovisioned requests cause the cluster autoscaler to provision more nodes than the workload needs, because nodes appear full based on scheduled requests rather than real consumption. The Cast AI 2026 State of Kubernetes Optimization Report found 69% of requested CPU goes unused across production clusters. That’s up from 40% the prior year. Memory overprovisioning sits at 79%. Reducing requests to match observed usage directly reduces the node count required to run the same workload. What are good default requests for Kubernetes containers? There is no universal default. Derive values from observed usage: set CPU requests at p95 of rate container cpu usage seconds total {container =””,container =”POD”} 5m over a 2-week window and set memory limits at p99 of container memory working set bytes with the same filters. Use a LimitRange to inject namespace-level defaults for example, 100m CPU and 128Mi memory as defaultRequest so that pods without explicit specs still have reasonable values. Revisit settings every one to two months, because workload behavior changes as applications evolve.