Kubernetes Pending Pod: CPU Requests, Unbound PVC, Taints
The scheduler event names every node and its reason. Requests versus allocatable, untolerated taints, zonal volume binding, and impossible anti-affinity.

Introduction & Quick Summary
Symptom. A pod stays Pending and the rollout never finishes. kubectl describe pod ends with Events: Warning FailedScheduling 0/6 nodes are available: 3 Insufficient cpu, 2 node(s) had untolerated taint {dedicated: gpu}, 1 node(s) had volume node affinity conflict. Or the message is shorter: 0/6 nodes are available: pod has unbound immediate PersistentVolumeClaims.
Cause. Pending is a scheduler verdict, not an application failure, and the event string lists every node with its own reason. The common four are: the sum of requests (not usage) of all pods on a node leaves no room for this one; a taint on the eligible nodes has no matching toleration; a requiredDuringSchedulingIgnoredDuringExecution affinity rule matches no node; and the PersistentVolumeClaim is not bound, either because no PersistentVolume exists or because its zone does not match any schedulable node.
Fix. Read the event string node by node before changing anything. Then either lower the request to something real, add the toleration, remove the hard affinity, or fix the claim — and remember that with volumeBindingMode: WaitForFirstConsumer the claim binds only after scheduling, so a nodeSelector narrow enough to exclude the volume's zone deadlocks.
Core Mechanics, Signatures & Reproducible Setup
The scheduler filters nodes in two phases: Filter (does this node satisfy requests, taints, affinity, volumes?) and Score. A pod that passes no node's Filter stays Pending with a per-node reason string.
kubectl describe pod web-7d9f8b6c5-x2k4p | sed -n '/Events:/,$p'
kubectl get events --field-selector involvedObject.name=web-7d9f8b6c5-x2k4p \
--sort-by=.lastTimestamp# a deliberately over-requesting pod: reproduces Insufficient cpu on a small cluster
apiVersion: v1
kind: Pod
metadata:
name: requests-too-high
spec:
containers:
- name: app
image: nginx:1.27-alpine
resources:
requests: { cpu: "2", memory: "4Gi" }
limits: { cpu: "2", memory: "4Gi" }With 3 × 2-vCPU nodes and any other workload present, this pod reports 3 Insufficient cpu and never schedules. Change requests.cpu to 200m and it runs immediately — which proves the request, not the limit, was the blocker.
Key Comparison Table
| Event fragment | Means | First check |
|---|---|---|
Insufficient cpu | node's allocatable minus sum of requests | kubectl describe node allocated resources |
Insufficient memory | same, for memory | largest resident requests, not usage |
unbound immediate PersistentVolumeClaims | claim not bound | kubectl get pvc -o wide |
volume node affinity conflict | PV zone excludes all nodes | PV nodeAffinity vs node labels |
untolerated taint {key: value} | no toleration | kubectl get nodes -o custom-columns |
node(s) didn't match Pod's node affinity/selector | labels do not match | kubectl get nodes --show-labels |
node(s) didn't match pod anti-affinity rules | replica co-location blocked | count nodes vs replicas |
Preemption is not helpful | nothing to evict | priority class |
| Volume binding mode | Binds when | Deadlock risk |
|---|---|---|
Immediate | at PVC creation | claim bound in a zone with no node |
WaitForFirstConsumer | after the pod is scheduled | nodeSelector excludes the volume's zone |
pre-bound PV (volumeName) | immediately | PV already in the wrong zone |
Practical Edge Cases, Error Fixes & Deep Dive Code Examples
1. Reading the node capacity that matters
requests are reserved whether or not the process uses them, and only allocatable counts — the kubelet's reserved resources are not available to pods.
kubectl describe node ip-10-0-1-24 | sed -n '/Allocated resources/,/Events/p'Allocated resources:
Resource Requests Limits
cpu 1850m (92%) 4 (200%)
memory 3.1Gi (81%) 6Gi (156%)92% of CPU requests means a pod asking for 200m cannot fit even though the node reports low real usage. Two fixes: lower the request to a measured value, or add a node. Deleting other pods to free requests is only correct if those pods are genuinely unnecessary.
2. Taints and the node that was supposed to be excluded
A GPU node pool tainted dedicated=gpu:NoSchedule will not accept ordinary pods, and the scheduler reports 2 node(s) had untolerated taint. Adding a toleration lets the pod land there, which is usually the wrong outcome — it consumes an expensive node. Prefer nodeSelector on a CPU pool, and add tolerations only for daemons that must run everywhere.
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoScheduleNoSchedule blocks new pods; NoExecute also evicts running ones, which is why adding a NoExecute taint causes a restart wave rather than a scheduling failure.
3. PVC that never binds
Three distinct causes share one symptom, and the PVC's events name which:
kubectl describe pvc data-postgres-0 | sed -n '/Events:/,$p'no persistent volumes available for this claim and no storage class is set—storageClassNamemissing or misspelled.waiting for first consumer to be created before binding— normal withWaitForFirstConsumer, not an error; check whether the pod was scheduled at all.provisioned volume ... zone us-east-1atogether with nodes only in1band1c— the class saysImmediateand provisioned in the wrong zone.
# the usual correct shape on a zonal cloud
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gp3
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: trueWith that class the volume is created in the zone the pod actually scheduled into, so the conflict cannot arise.
4. Anti-affinity with fewer nodes than replicas
requiredDuringSchedulingIgnoredDuringExecution on topologyKey: kubernetes.io/hostname with replicas: 5 and 3 nodes leaves two pods permanently Pending — by design. Switch to preferredDuringSchedulingIgnoredDuringExecution, or use topologySpreadConstraints with whenUnsatisfiable: ScheduleAnyway.
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels: { app: web }5. ResourceQuota and LimitRange rejections
A LimitRange can rewrite a missing request to a default that no node can satisfy, and a ResourceQuota blocks creation with exceeded quota: compute-resources, requested: requests.cpu=2, used: requests.cpu=7, limited: requests.cpu=8. The pod object exists and is Pending with no scheduling event, which is easy to misread.
kubectl describe quota -n prod
kubectl get events -n prod --field-selector reason=FailedCreate6. Preemption and priority classes
When nothing fits, the scheduler can evict lower-priority pods if the incoming pod's priorityClassName is higher. Preemption is not helpful for scheduling means even evicting everything on the best node does not make room — a signal that the request itself is impossible, not that the cluster is full.
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata: { name: critical }
value: 1000000
globalDefault: falsePreemption is disruptive to whatever it evicts; it is a tool for a genuinely critical workload, not for a web deployment that should simply have a realistic request.
Common Compilation/Runtime Mistakes & Version Gotchas
- Reading
kubectl topto explainInsufficient cpu. The scheduler uses requests; usage is irrelevant to it. requestscopied from a VM's size. A2CPU request on a 2-vCPU node leaves room for nothing else.NoExecutetaints added during maintenance. They evict running pods, and tolerations must already be deployed.Immediatebinding on a zonal storage class. The volume can be provisioned in a zone with no schedulable node.nodeSelectorplusWaitForFirstConsumerpointing at different zones. PermanentPendingwith two separate reasons.- Anti-affinity
requiredwith more replicas than nodes. Mathematically impossible; usepreferredor spread constraints. - A
LimitRangedefault larger than any node's allocatable. Every pod without explicit requests becomes unschedulable.
Related tools. Normalise the pod's full JSON status before diffing it against a working replica with the JSON formatter, diff two manifests to spot the affinity or request that changed with the text diff, and convert the event timestamps to local time with the timestamp converter.
Try matching browser tools
All toolsSmall utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Kubernetes guides
All Kubernetes guidesKubernetes Exit Code 137 OOMKilled vs Exit Code 134 Heap Limits
137 is SIGKILL from the cgroup or a liveness probe; 134 is an abort, usually V8's own heap ceiling. Telling them apart and sizing the two limits together.
CrashLoopBackOff: Reading Exit Codes and Debugging Restart Loops
The backoff is only the waiting state. The evidence is in Last State, the exit code, and the previous container's last log line, and each points to a different fix.
ImagePullBackOff and ErrImagePull: Reading the Real Pull Error
ImagePullBackOff hides five different failures. The event text names which one: wrong reference, missing credentials, wrong architecture, unreachable registry, or a rate limit.
kubectl Output: jsonpath, custom-columns, and the Debugging Commands Worth Knowing
jsonpath for one value in a script, custom-columns for tables people read, and kubectl debug when the container has no shell. Includes the three quoting layers that break jsonpath.
More in Kubernetes
All Kubernetes guidesThe guides filed next to this one in Kubernetes, in library order.