KubernetesHigh
Fix: Kubernetes Pod Stuck in Pending State
Pod stays in Pending state indefinitely
!Symptoms
- Pod stays in Pending state indefinitely
- kubectl describe pod shows 'Unschedulable' or 'Insufficient' events
- No node has enough resources to schedule the pod
- Pod is waiting for a PersistentVolume to be bound
?Root Causes
- Insufficient CPU or memory on available nodes
- Node selector, affinity, or taint/toleration mismatch
- PersistentVolumeClaim cannot be bound (no matching PV or storage class)
- Resource quotas exceeded in the namespace
- Cluster autoscaler not configured or scaling too slowly
#Diagnosis Steps
- 1Describe the pod: `kubectl describe pod <pod-name>` and check Events
- 2Check node resources: `kubectl describe nodes | grep -A 5 'Allocated resources'`
- 3Check PVC status: `kubectl get pvc` — look for Pending PVCs
- 4Check resource quotas: `kubectl describe resourcequota -n <namespace>`
- 5Check node taints: `kubectl describe nodes | grep Taints`
>Fix
- 1Reduce resource requests in the pod spec to fit available nodes
- 2Add or resize nodes in the cluster to provide more capacity
- 3Fix node selector/affinity rules to match available nodes
- 4Add tolerations for node taints that are blocking scheduling
- 5Create the required PersistentVolume or fix the StorageClass
- 6Increase namespace resource quotas if they are too restrictive
*Prevention
- Enable Cluster Autoscaler to automatically add nodes when needed
- Set appropriate resource requests (not too high, not too low)
- Use Pod Disruption Budgets to prevent over-packing nodes
- Monitor cluster capacity and set alerts at 80% utilization
- Use LimitRanges to set default resource requests for pods
Related Error Messages
0/3 nodes are availableInsufficient cpuInsufficient memorypod has unbound immediate PersistentVolumeClaimsFailedSchedulingdidn't match Pod's node affinity/selector