EC
Pending (Insufficient cpu)
KubernetesWARNINGNotableSchedulingHIGH confidence

Pod cannot be scheduled due to insufficient CPU

Production Risk

If pods for a critical service cannot be scheduled, it can lead to failed deployments, inability to scale out, and service degradation or outages.

What this means

The Kubernetes scheduler is unable to find a node with enough available CPU resources to satisfy the pod's CPU request. The pod will remain in a pending state until resources become available.

Why it happens
  1. 1The pod requests more CPU than is available on any single node in the cluster
  2. 2Existing pods on all nodes have consumed all available CPU, leaving no room for the new pod
  3. 3Node taints or affinity rules are restricting the pod to a subset of nodes that are already full
How to reproduce

A newly created pod remains in the Pending state. `kubectl describe pod` reveals the scheduling failure reason.

trigger — this will error
trigger — this will error
kubectl describe pod high-cpu-pod

expected output

Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  2m (x12 over 5m)     default-scheduler  0/3 nodes are available: 3 Insufficient cpu.

Fix 1

Check node capacity and allocation

WHEN To see how much CPU is available on each node

Check node capacity and allocation
kubectl describe nodes | grep -E "Name:|Allocatable:|cpu"

Why this works

This command sequence lists each node's name and its allocatable CPU, providing a quick overview of cluster capacity to compare against the pod's request.

Fix 2

Lower the pod's CPU request

WHEN The requested CPU is higher than necessary or higher than any node can provide

Lower the pod's CPU request
kubectl patch deployment my-app-deployment -p '{"spec":{"template":{"spec":{"containers":[{"name":"my-app","resources":{"requests":{"cpu":"250m"}}}]}}}}'

Why this works

This command reduces the pod's CPU request to a smaller value, which may allow the scheduler to find a suitable node.

Fix 3

Add a new node to the cluster

WHEN The cluster is genuinely at capacity and more CPU resources are needed

Add a new node to the cluster
gcloud container clusters resize my-cluster --num-nodes=4

Why this works

For cloud-managed clusters, the provider's command-line interface can be used to add more nodes, increasing the total available CPU in the cluster.

What not to do

✕ Remove CPU requests from all pods

This makes pods 'best effort' and leads to unpredictable performance and potential CPU starvation for critical workloads. It also makes scheduling less efficient.

Sources
Official documentation ↗

k8s.io/kubernetes/pkg/scheduler/framework/plugins/noderesources/fit.go

Troubleshooting Pods Pending ↗

Content generated with AI assistance and reviewed for accuracy. Found an error? hello@errcodes.dev

← All Kubernetes errors