Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Kubernetes Pods: Lifecycle Phases, Init Containers, and Restart Policies

Phase versus container state, ordered init containers that rerun on restart, native sidecars, and the exit codes behind CrashLoopBackOff. How to read a pod's status in the order that answers the question.

~6 min readKubernetes
#Kubernetes#Pods#Operations
3D illustration of a pod cube with ordered init container stages leading into running application containers

Quick Summary

A Pod is the smallest schedulable unit in Kubernetes: one or more containers sharing a network namespace, an IPC namespace, and volumes, scheduled together onto one node. Its lifecycle is described at two levels. The pod status.phase is a coarse summary — Pending, Running, Succeeded, Failed, Unknown — while each container has a state (Waiting, Running, Terminated) with a reason such as ImagePullBackOff, CrashLoopBackOff, or OOMKilled. The phase is almost never the useful diagnostic; the container state reasons are.

Init containers run before any app container, one at a time, each to completion, with their own image and their own volume mounts. They exist for setup that must not run concurrently with the app: waiting for a database, cloning a config repository, fixing volume permissions, or generating a TLS key. Because they run in the same pod, they see the same network and the same volumes, and their logs are readable with -c.

restartPolicy is per pod, not per container: Always (default, for workloads), OnFailure, or Never (for one-shot jobs). The kubelet applies it with exponential backoff capped at five minutes, which is why a crashing container shows CrashLoopBackOff with growing gaps rather than a tight loop. This guide covers the phase and condition model, init containers and native sidecars, restart and termination semantics, and the status queries that answer "why is this pod not running".

Basic Syntax & Core Signatures

yaml
apiVersion: v1 kind: Pod metadata: name: api labels: { app: api } spec: restartPolicy: Always # Always | OnFailure | Never terminationGracePeriodSeconds: 30 activeDeadlineSeconds: 600 # hard cap on total runtime initContainers: - name: wait-for-db image: busybox:1.36 command: ['sh', '-c', 'until nc -z db 5432; do echo waiting; sleep 2; done'] - name: fix-perms image: busybox:1.36 command: ['sh', '-c', 'chown -R 1000:1000 /data'] volumeMounts: [{ name: data, mountPath: /data }] containers: - name: api image: ghcr.io/acme/api:1.4.2 ports: [{ containerPort: 8080 }] volumeMounts: [{ name: data, mountPath: /data }] volumes: - name: data emptyDir: {}
bash
# Lifecycle and state kubectl get pod api -o wide kubectl get pod api -o jsonpath='{.status.phase}{"\n"}{range .status.containerStatuses[*]}{.name}={.state}{"\n"}{end}' kubectl describe pod api # events at the bottom: the real story kubectl logs api -c wait-for-db # init container log kubectl logs api --previous # the container that died # Conditions kubectl get pod api -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}' # PodScheduled=True Initialized=True ContainersReady=True Ready=True # Wait helpers kubectl wait --for=condition=Ready pod/api --timeout=120s kubectl wait --for=jsonpath='{.status.phase}'=Succeeded job/migrate --timeout=300s
yaml
# Native sidecar (Kubernetes 1.29+, stable in 1.33): an init container that never exits initContainers: - name: log-shipper image: fluent/fluent-bit:3.0 restartPolicy: Always volumeMounts: [{ name: logs, mountPath: /var/log/app }]
yaml
# Lifecycle hooks lifecycle: postStart: exec: { command: ["/bin/sh", "-c", "curl -sf localhost:8080/ready || true"] } preStop: exec: { command: ["/bin/sh", "-c", "sleep 5; /app/drain.sh"] }

Key Comparison Table

PhaseMeaningTypical cause
PendingAccepted, not yet runningUnschedulable, image pulling, init container running
RunningBound to a node, at least one container runningMay still be Ready=False
SucceededAll containers exited 0, and will not restartJob, OnFailure/Never
FailedAll containers terminated, at least one failedBad command, activeDeadlineSeconds
UnknownNode not reportingNode NotReady, kubelet unreachable
Container state / reasonMeaningFirst thing to check
Waiting / ImagePullBackOffCannot pull the imageImage name, tag, registry secret
Waiting / CrashLoopBackOffKeeps exiting, backoff appliedlogs --previous, exit code
Running / Ready=FalseFailing readinessReadiness probe, dependency health
Terminated / OOMKilledExceeded memory limitresources.limits.memory, heap settings
Terminated / CompletedExit code 0Normal for a job
Terminated / ErrorNonzero exitEntrypoint, config, missing file
Restart policyApplies toExit 0Exit nonzero
AlwaysDeployments, StatefulSets, DaemonSetsRestartsRestarts
OnFailureJobsStays CompletedRestarts
NeverJobs, debugging podsStays CompletedStays Failed
FieldEffect
terminationGracePeriodSecondsTime between SIGTERM and SIGKILL
activeDeadlineSecondsKills the pod after this many seconds
shareProcessNamespaceContainers see each other's processes
initContainers[].restartPolicy: AlwaysSidecar that starts first and stays running

Practical Edge Cases & Deep Dives

1. Pending with no events: the scheduler has nothing to say

bash
$ kubectl describe pod api | tail -5 Events: Warning FailedScheduling 30s (x5) default-scheduler 0/3 nodes are available: 3 Insufficient cpu.

Pending splits into two causes. Either the scheduler cannot place the pod (FailedScheduling, with a reason such as insufficient CPU/memory, no matching node selector or affinity, taints without tolerations, or an unbound PVC), or it is placed and something else blocks start (ImagePullBackOff, a volume that cannot attach, or a long init container). The event text names which. Scheduling problems repeat with the same reason and only appear when the pod is re-queued; wait for the second event if the first is empty.

bash
kubectl get events --field-selector involvedObject.name=api --sort-by=.lastTimestamp kubectl describe node | grep -A5 'Allocated resources'

2. Init containers: ordered, blocking, and re-run on restart

bash
$ kubectl get pod api -o jsonpath='{.status.initContainerStatuses[*].name}' wait-for-db fix-perms $ kubectl logs api -c wait-for-db waiting waiting

Init containers run strictly in order and the pod does not start app containers until all have succeeded. Two consequences: a bad init container makes the pod look stuck with no application logs, and if the pod restarts, every init container runs again (restartPolicy: Always re-runs them on each pod restart, not on container restart within the pod). A long-running init container (waiting for a database) therefore delays every restart, which is why activeDeadlineSeconds on the pod or a bounded retry loop inside the init container is worth adding. Init containers support a subset of fields — no lifecycle, no readinessProbe, no ports — and consume resources in the pod's scheduling calculation based on the largest request of any single container.

3. Native sidecars, and why they are init containers

yaml
initContainers: - name: proxy image: envoyproxy/envoy:v1.31.0 restartPolicy: Always

Before native sidecars, a sidecar was an ordinary container in the pod, which meant it could start after the app, could be the only thing keeping a completed Job's pod alive, and could not be ordered relative to init containers. A restartPolicy: Always init container starts in init order, keeps running, stays alive with the pod, does not block job completion in the same way, and stops in reverse order at shutdown. Istio and other meshes moved to this form; it requires a cluster at 1.29+ (beta) or 1.33+ (stable), and kubectl older than that version will show it puzzlingly.

4. CrashLoopBackOff and the exit code that explains it

bash
$ kubectl get pod api NAME READY STATUS RESTARTS AGE api 0/1 CrashLoopBackOff 5 6m $ kubectl get pod api -o jsonpath='{.status.containerStatuses[0].lastState.terminated}' {"exitCode":1,"reason":"Error","startedAt":"...","finishedAt":"..."}

Exit 1 is the application's own failure; 137 means SIGKILL (an OOM kill or a probe-forced restart); 143 means SIGTERM (graceful termination, normal during rollout); 139 is a segfault. CrashLoopBackOff itself is just an event count — the backoff starts at 10 seconds, doubles, and caps at 5 minutes, and the container is recreated regardless until kubectl delete or the spec changes. Always read --previous logs: the current container may have produced no output yet.

bash
kubectl logs api --previous --tail=100 kubectl get pod api -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}{"\n"}'

5. Termination order and the requests that never finish

yaml
spec: terminationGracePeriodSeconds: 60 containers: - name: api lifecycle: preStop: exec: { command: ["/bin/sh", "-c", "sleep 10"] }

On deletion the pod goes to Terminating: the endpoint is removed from the Service (in parallel), preStop runs, then SIGTERM is sent, then SIGKILL after the grace period. The classic bug is a preStop sleep longer than the grace period, which guarantees SIGKILL, or an app that ignores SIGTERM and relies on grace that other containers already used. A preStop sleep of a few seconds is a pragmatic way to cover the window between endpoint removal and the proxy updating, but the correct pattern is an application that drains on SIGTERM and exits well inside terminationGracePeriodSeconds.

Note that a pod stuck in Terminating for hours is usually a node-level problem (a volume that will not unmount, or a kubelet that stopped responding); kubectl delete pod --grace-period=0 --force leaves the volume attached and should be the last resort.

6. Debugging without modifying the workload

bash
# Ephemeral debug container: shares the pod's namespaces kubectl debug -it pod/api --image=nicolaka/netshoot --target=api # A copy of the pod with a different command, without touching the original kubectl debug pod/api --copy-to=api-dbg --container=api -- sh -c 'sleep 3600' # From a node's namespaces kubectl debug node/worker-1 -it --image=busybox

kubectl debug --target requires the target container's process namespace and a distribution image with the tools; --copy-to is the safer route when the pod spec is immutable or managed by a controller. Both leave objects behind, so delete the debug pod and the ephemeral container's effects explicitly.

Common Runtime & Configuration Mistakes

SymptomCauseFix
Pending forever, no eventsScheduler has not re-queued, or a webhook blocks creationCheck describe, validatinog webhooks, ResourceQuota
FailedScheduling: Insufficient cpuRequests exceed the node's allocatableLower requests, add nodes, check Allocated resources
ImagePullBackOffWrong tag or missing imagePullSecretsFix the reference, create the secret in the namespace
CreateContainerConfigErrorReferenced ConfigMap/Secret key missingCreate the key or the object
Init:CrashLoopBackOffInit container failing repeatedlylogs -c <init>, its exit code
App never starts, init container slowOrdered, blocking initBound the wait, add activeDeadlineSeconds
OOMKilled (exit 137)Memory limit below the working setRaise the limit, set runtime heap/max-old-space
Pod restarts read as a deployment bugrestartPolicy: Always with a failing appFix the app; restartPolicy is a pod-level policy
Job pod never completesSidecar never exitsNative sidecar or restartPolicy: Never
Long Terminating, then SIGKILLpreStop longer than the grace periodShorten preStop, raise the grace period
Terminating stuck for hoursVolume unmount or node problemDrain the node; do not force-delete first
Exec into a distroless container failsNo shell in the imagekubectl debug --target with a tool image

A pod's status is a story in layers: the phase says whether the scheduler and kubelet moved it, the container states name the reason, the events carry timestamps and messages, and the logs (including --previous and per-init-container) carry the application's side. Reading them in that order resolves almost every "it's not starting" question without changing a single line of the manifest.

Try matching browser tools

All tools

Small utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Kubernetes guides

More in Kubernetes

All Kubernetes guides

The guides filed next to this one in Kubernetes, in library order.