Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Helm: another operation (install/upgrade/rollback) is in progress

Helm 3 stores the lock inside the release secret. Read the pending status, roll back to the last good revision, or clear the record deliberately.

~5 min readKubernetes
#Helm#Kubernetes#CI/CD#Troubleshooting
A helm release stack locked by a release block, an amber frozen gear and a cyan rollback arrow

Introduction & Quick Summary

Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress is Helm refusing to start because the release's recorded status is still pending-install, pending-upgrade or pending-rollback. Helm 3 has no lock server and no lease: the release Secret is the lock. If a previous operation did not write a final status, every later operation sees a release that looks busy and stops.

The status usually got stuck for one of these reasons:

  • The previous command was interrupted. Ctrl+C on helm upgrade --wait, a cancelled CI job, or an evicted runner leaves the pending record behind.
  • A pre-upgrade hook never finished. A Job that waits for a database that is not reachable keeps Helm inside the upgrade until the timeout expires.
  • Two pipelines ran at once. A push to main and a manual redeploy overlapped, and the second one started before the first wrote its status.
  • The timeout was shorter than the rollout. --timeout defaults to five minutes; a slow image pull plus a readiness gate easily exceeds it.
Status shownWhat it meansSafe next action
pending-installA first install never completedFix the cause, then uninstall or delete the pending secret
pending-upgradeAn upgrade was interruptedRoll back to the last good revision, or clear the record
pending-rollbackA rollback was interruptedRoll back again after clearing, or to another revision
failedThe operation finished and failedhelm rollback, not a delete
deployedHealthyNothing to do

Core Mechanics, Signatures & Reproducible Setup

Every Helm 3 release is stored as a Secret named sh.helm.release.v1.<release>.v<revision> in the release namespace, labelled owner=helm. The payload is the release object, gzipped and base64-encoded, with a status field. helm upgrade reads the latest revision, refuses if its status is pending, and only writes deployed or failed when it finishes.

That design explains two behaviours people find surprising. First, the "lock" is per release name and namespace, so two releases in different namespaces never block each other. Second, deleting a pod or restarting the cluster does not clear a pending status, because there is no process to notice: the record stays until something writes over it.

A reproducible stuck release needs only a hook that sleeps longer than the timeout:

# templates/hook.yaml apiVersion: batch/v1 kind: Job metadata: name: migrate annotations: "helm.sh/hook": pre-upgrade "helm.sh/hook-delete-policy": before-hook-creation spec: template: spec: restartPolicy: Never containers: - name: migrate image: yourorg/migrate:1.4.0 command: ["sh", "-c", "sleep 900 && ./migrate"]

Then:

bash
helm upgrade api ./charts/api -n apps --wait --timeout 2m
text
Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress

The next run fails immediately, before contacting the cluster, because the status from the previous run is still pending-upgrade.

Key Comparison Table

Clearing the state has three routes with different consequences, and picking the wrong one loses history or leaves resources orphaned.

RouteCommandKeeps historyUse when
Roll backhelm rollback api <rev> -n appsYes, new revisionAn earlier revision is known good
Delete pending secretkubectl delete secret sh.helm.release.v1.api.v7 -n appsYes, minus that revisionThe pending revision is garbage
Uninstallhelm uninstall api -n appsRemovedReinstalling from scratch is acceptable

The lifecycle that produces a pending status in practice, compared with a healthy one:

SituationStatus while runningStatus after successStatus after interruption
First installpending-installdeployedStays pending-install
Upgradepending-upgradedeployedStays pending-upgrade
Rollbackpending-rollbackdeployedStays pending-rollback
Upgrade with --atomicpending-upgradedeployedRolls back automatically

Practical Edge Cases, Error Fixes & Deep Dive Code Examples

Read the release before touching it

Two commands answer almost every case. helm status shows the current status and the last description, and helm history shows every revision with its status, which tells you which revision to roll back to.

bash
helm status api -n apps helm history api -n apps --max 10 kubectl get secret -n apps -l owner=helm,name=api \ --sort-by=.metadata.creationTimestamp

The secret list is the ground truth: the highest v<N> is the pending revision, and its metadata.creationTimestamp tells you when the interrupted run happened. A pending revision from three weeks ago is stale; one from two minutes ago may still have a running hook.

Roll back instead of deleting

If the last deployed revision is healthy, a rollback writes a new revision with status deployed and clears the pending one. Helm allows this even while the release is pending, which is why it is the preferred first attempt.

bash
helm rollback api 6 -n apps --wait --timeout 5m helm history api -n apps --max 5
text
REVISION STATUS DESCRIPTION 8 deployed Rollback to 6 7 pending-upgrade Upgrade "api" failed: context canceled 6 superseded Upgrade complete

If helm rollback also reports "another operation in progress", the pending revision cannot be superseded, and deleting the secret is the next step.

Clear a pending revision deliberately

Deleting the pending release secret removes the lock. Helm then continues from the last deployed revision, so the next upgrade works normally. Do it only when no Helm process is actually running: check kubectl get pods -n apps for a hook Job still executing, and helm list -n apps --all for the release.

bash
kubectl delete secret sh.helm.release.v1.api.v7 -n apps helm list -n apps # status should be deployed (revision 6) helm upgrade api ./charts/api -n apps --wait --timeout 5m

Never delete the whole sh.helm.release.v1.api.* set: that erases the release history and makes Helm treat the release as unmanaged while its resources still exist.

Stop creating the problem: atomic upgrades and CI concurrency

--atomic makes a failed upgrade roll itself back, so an interrupted rollout does not leave the release pending, and --timeout should be set from the rollout's real duration rather than the default.

bash
helm upgrade --install api ./charts/api -n apps \ --atomic --wait --timeout 10m --history-max 10

On the pipeline side, serialise the job per environment so two runs cannot overlap:

concurrency: group: helm-${{ github.ref_name }} cancel-in-progress: false

cancel-in-progress: true is the wrong choice here: cancelling mid-upgrade is one of the ways a release ends up pending in the first place.

When the hook itself is the lock

A pre-upgrade hook that never completes keeps the release pending, and no amount of clearing helps until the hook is fixed. Find it, then give it a deadline:

spec: backoffLimit: 1 activeDeadlineSeconds: 300

and make Helm remove it on the next attempt with "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded. Without the second policy, a completed Job stays and the next upgrade waits on it in some Helm versions.

Common Compilation/Runtime Mistakes & Version Gotchas

Running helm uninstall as the first reaction. It deletes the workloads and keeps the release record, so the reinstall starts from pending-install again. Roll back or clear the pending secret first, unless a full teardown is what you actually want.

Assuming a cluster restart clears the status. It does not; the pending status lives in the Secret. Check helm history after a node failure instead of retrying the same command.

Using --force. --force replaces resources via delete-and-recreate and is not a lock workaround. It can leave pods down and still hit the pending check.

Confusing the namespace. The release secret, the hook Job and the workloads can live in different namespaces. helm status without -n looks in the current context's namespace, which is where the "release not found" and "another operation" errors get mixed up.

Ignoring --wait semantics. Without --wait, Helm returns as soon as the API accepts the objects, so a failed rollout looks successful and the release is deployed while pods crash. With --wait, a slow rollout can exceed the timeout and leave pending-upgrade. Neither default is right for every chart; set both explicitly.

Version behaviour. Helm 3 introduced the Secret-based release storage; Helm 2 stored releases in ConfigMaps under Tiller, where the message was different. The pending-* statuses, --atomic and --history-max all behave as described on Helm 3.x.

The pending status is a record of an unfinished transaction, so the question to answer first is always whether that transaction is still running. When you edit values or hook manifests, the YAML validator catches the indentation errors that make a hook hang silently, and the Base64 encoder helps inspect a release Secret's gzipped payload.

Official references: helm rollback and helm upgrade.

Official references

primary documentation

The Kubernetes documentation that defines the behaviour this guide explains.

Try matching browser tools

All tools

Small utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Kubernetes guides

More in Kubernetes

All Kubernetes guides

The guides filed next to this one in Kubernetes, in library order.