Kubernetes Jobs and CronJobs: Completions, Parallelism, Backoff, and Missed Schedules
completions counts successes, parallelism counts concurrency, and backoffLimit counts retries. Plus podFailurePolicy, restartPolicy differences, and the CronJob timezone trap.

Quick Summary
A Job runs pods until a required number of them exit successfully; a CronJob creates Jobs on a schedule. The counters are explicit: completions is how many successful pod exits finish the Job, parallelism is how many run at once, and backoffLimit is how many failures are tolerated before the Job is marked Failed.
restartPolicy decides who retries. With Never, a failed pod stays behind and the Job controller creates a replacement, so pod count grows and logs of the failed attempt survive. With OnFailure, the kubelet restarts the container inside the same pod, so backoffLimit counts container restarts and the pod list stays small. Neither policy retries after a successful exit; that is the contract that makes a Job terminable at all.
A CronJob adds scheduling on top: a five-field cron expression, a timezone, concurrencyPolicy for overlapping runs, startingDeadlineSeconds for late starts, and history limits that control how many finished Jobs linger. The failure that bites most teams is not cron syntax but a missed schedule evaluated against the controller's last-seen time, and jobs that overlap because a previous run had not finished.
Basic Syntax & Core Signatures
apiVersion: batch/v1
kind: Job
metadata:
name: reindex-20261115
spec:
completions: 4 # successful pods required (default 1)
parallelism: 2 # pods running simultaneously (default 1)
backoffLimit: 3 # retries before Failed (default 6)
activeDeadlineSeconds: 1800 # wall clock cap; exceeded => Failed
ttlSecondsAfterFinished: 3600 # delete the Job an hour after it finishes
completionMode: Indexed # Indexed | NonIndexed
podFailurePolicy: # optional: fail fast on specific exits
rules:
- action: Fail
onExitCodes:
operator: In
values: [42]
template:
metadata:
labels: { job: reindex }
spec:
restartPolicy: Never # Never | OnFailure
containers:
- name: reindex
image: registry.example.com/search/reindex:2.4.1
args: ["--shard=$(JOB_COMPLETION_INDEX)"]
resources:
requests: { cpu: 500m, memory: 512Mi }apiVersion: batch/v1
kind: CronJob
metadata:
name: nightly-report
spec:
schedule: "0 3 * * *" # minute hour day-of-month month day-of-week
timeZone: "Asia/Seoul" # stable since 1.27; default is UTC
concurrencyPolicy: Forbid # Allow | Forbid | Replace
startingDeadlineSeconds: 300 # skip if it could not start within 5 min
suspend: false
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
backoffLimit: 2
ttlSecondsAfterFinished: 86400
template:
spec:
restartPolicy: OnFailure
containers:
- name: report
image: registry.example.com/reports/nightly:1.2.0kubectl create job --from=cronjob/nightly-report nightly-report-manual
kubectl wait --for=condition=complete job/reindex-20261115 --timeout=1800s
kubectl logs job/reindex-20261115 --all-containers
kubectl get pods -l job-name=reindex-20261115 # one pod per attempt with NeverKey Comparison Table
| Setting | restartPolicy: Never | restartPolicy: OnFailure |
|---|---|---|
| Retry location | New pod per failure | Container restart in place |
backoffLimit counts | Failed pods | Container restarts |
| Failed pod left behind | Yes (logs preserved) | No |
| Best for | Long batch jobs, debugging | Cheap idempotent jobs |
podFailurePolicy | Applies to pod deletion/failure | Applies, matches container exit |
| CronJob field | Default | Effect |
|---|---|---|
concurrencyPolicy | Allow | Forbid skips a tick if the last run is active; Replace kills it |
startingDeadlineSeconds | unset | A run that cannot start in time is counted as missed |
successfulJobsHistoryLimit | 3 | Completed Jobs kept |
failedJobsHistoryLimit | 1 | Failed Jobs kept |
suspend | false | Stops new runs without deleting the CronJob |
timeZone | UTC (node local before 1.27) | Schedule interpretation |
| Symptom | Meaning |
|---|---|
BackoffLimitExceeded | Retries exhausted; read the failed pod's log |
DeadlineExceeded | activeDeadlineSeconds reached |
Job Running but no pods | Quota or unschedulable pod |
| CronJob ran twice at once | concurrencyPolicy: Allow with a slow job |
| CronJob never fires | Invalid schedule, suspended, or past missed-run threshold |
Practical Edge Cases & Deep Dives
1. completions, parallelism, and the work queue pattern
With completions: 4, parallelism: 2, the controller runs two pods, and as each succeeds it starts another until four successes accumulate. That is the fixed-amount pattern. The work-queue pattern drops completions entirely and sets only parallelism: the Job succeeds when at least one pod exits 0 and no pods are running, which suits workers that pull from a queue until it is empty.
# Completions count successes, not pods: index 1 failing twice still needs 4 successes
kubectl get pods -l job-name=reindex-20261115
# reindex-20261115-0-abcde 0/1 Error
# reindex-20261115-0-fghij 0/1 Completed <- success 1 of 4completionMode: Indexed gives each pod JOB_COMPLETION_INDEX from 0 to completions-1 and a stable hostname (<job>-<index> in a headless Service), which is how a Job partitions 100 inputs across 10 workers without a queue. In indexed mode completions is required, parallelism may be lower than it, and a failed index is retried as the same index.
2. What backoffLimit actually counts, and the retry ladder
Retries use exponential backoff between 10 seconds and 6 minutes (the delay doubles per failure up to the cap), and backoffLimit: 6 is the default. With restartPolicy: Never, every attempt is a new pod, so kubectl get pods shows one pod per attempt, and the failed ones remain for inspection unless ttlSecondsAfterFinished or a manual delete removes them.
$ kubectl describe job reindex-20261115 | tail -5
Warning BackoffLimitExceeded Job has reached the specified backoff limit
$ kubectl get job reindex-20261115 -o jsonpath='{.status.failed}{"\n"}'
4A misconfigured backoffLimit: 0 means one failure fails the Job immediately — useful for a migration that must not retry, and a trap when a transient 429 is expected. A pod evicted or lost with its node also consumes the budget.
3. podFailurePolicy: distinguishing retryable from fatal
podFailurePolicy:
rules:
- action: Ignore # don't count against backoffLimit
onPodConditions:
- type: DisruptionTarget # eviction, preemption, drain
- action: Fail # stop immediately
onExitCodes:
operator: In
values: [2, 78]
- action: FailJob # mark the whole Job failed
onExitCodes:
operator: In
values: [137] # e.g. OOMKilled means "raise limits"Without a policy, exit 2 (bad input) burns the same budget as exit 137 (a transient OOM), and a batch Job spends twenty minutes retrying something that can never succeed. action: Ignore with onPodConditions: DisruptionTarget is the standard fix for Jobs that keep failing a drain.
4. CronJob timing: missed runs, overlap, and timezone
# Controller was down 09:00-11:00 for a "*/10 * * * *" schedule
# It wakes at 11:00 and evaluates the last 10 missed times.
# startingDeadlineSeconds unset => it starts one run immediately and
# counts the others as missed; more than 100 missed runs => no run at all.Three behaviours matter. A run whose start time is older than startingDeadlineSeconds is skipped rather than started late. If more than 100 schedules were missed while the controller was down, the CronJob logs Cannot determine if job needs to be started and does nothing until the next valid tick. And timeZone defaults to UTC for a CronJob spec, so 0 3 * * * runs at noon KST unless you set timeZone: Asia/Seoul — the single most common "it runs at the wrong hour" cause.
concurrencyPolicy: Forbid skips a tick when the previous Job is still running; Replace kills the running Job and starts a new one, which is right for a snapshot job and wrong for anything that must finish.
5. Jobs that never finish: sidecars, probes, and exit codes
A Job pod never completes when a sidecar keeps running after the main container exits — before native sidecars (Kubernetes 1.29+), the answer was to make the sidecar exit itself. A Job pod with a failing readiness probe is not failed either; it is simply not Ready, and the Job waits. And an exit code of 143 (SIGTERM) counts as failure, so a pod killed by activeDeadlineSeconds or an eviction consumes the backoff budget.
kubectl get pod -l job-name=reindex-20261115 -o jsonpath=\
'{range .items[*]}{.metadata.name}{"\t"}{.status.phase}{"\t"}{.status.containerStatuses[0].state.terminated.exitCode}{"\n"}{end}'Common Runtime & Configuration Mistakes
| Symptom | Cause | Fix |
|---|---|---|
Job stays Running forever | Sidecar never exits, or probe never passes | Native sidecar, or drop/relax the probe |
BackoffLimitExceeded on a permanent error | Retry budget spent on a fatal exit | podFailurePolicy with action: Fail |
Job Failed after one pod | backoffLimit: 0 | Raise it, or accept fail-fast |
DeadlineExceeded mid-work | Wall-clock cap too tight | Raise activeDeadlineSeconds; checkpoint work |
| Failed pods pile up | Never keeps every attempt | ttlSecondsAfterFinished, manual cleanup |
| CronJob runs at the wrong hour | timeZone unset (UTC) | Set timeZone explicitly |
| Two runs overlap | concurrencyPolicy: Allow | Forbid, or make the job idempotent |
| CronJob silently stops | >100 missed schedules | Fix controller availability; re-create the CronJob |
Forbidden: exceded quota on Job creation | Namespace quota counts Job pods | Raise count/pods in the ResourceQuota |
A Job is a completion contract, and every question about it reduces to three numbers: how many successes are required, how many failures are allowed, and whether anything is still running. When a batch workload misbehaves, read status.conditions, then the failed pod's exit code, then the retry budget in that order.
Try matching browser tools
All toolsSmall utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Kubernetes guides
All Kubernetes guideskubectl Output: jsonpath, custom-columns, and the Debugging Commands Worth Knowing
jsonpath for one value in a script, custom-columns for tables people read, and kubectl debug when the container has no shell. Includes the three quoting layers that break jsonpath.
Kubernetes Namespaces, ResourceQuotas, and LimitRanges
A namespace is only a boundary when quota and limit range make it one. Aggregate caps, per-pod defaults, the admission errors they produce, and namespaces stuck Terminating.
Kubernetes StatefulSets: Ordinal Identities, Headless Services, and PVC Lifecycle
Why each StatefulSet pod keeps a name, a DNS record, and its own PVC; how OrderedReady gates startup on readiness; and how partition-based updates canary a single quorum member.
Kubernetes securityContext, Capabilities, and Pod Security Standards
The five fields that remove most container escapes, which of them only exist at container level, and how the restricted profile enforces the same settings at admission.
More in Kubernetes
All Kubernetes guidesThe guides filed next to this one in Kubernetes, in library order.