Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Kubernetes ConfigMaps and Secrets: envFrom, Volume Mounts, and Immutability

Environment variables never update, mounted files do (after a minute), and subPath mounts never do. Plus the Secret types, the missing-key failures, and where credentials leak.

~6 min readKubernetes
#Kubernetes#Configuration#Security
3D illustration of config map file cards and a locked secret vault feeding env vars and volumes into a pod

Quick Summary

ConfigMaps hold non-secret configuration; Secrets hold data that should be handled more carefully. Both are namespaced API objects of key-value pairs, both can be mounted as files or injected as environment variables, and both are limited to 1 MiB because their content lives in etcd. The difference is intent plus a little enforcement: Secrets are base64-encoded (which is encoding, not encryption), can be restricted through RBAC, are mounted from memory-backed volumes, and are encrypted at rest only if the cluster's EncryptionConfiguration says so.

There are three ways to consume either object, and they behave differently at runtime. env.valueFrom injects one key and is never updated after the container starts. envFrom injects all keys (optionally with a prefix) and has the same limitation. A volume mount creates one file per key and is updated eventually — on the order of a minute, after the kubelet's sync and the API server's watch cache catch up — except when mounted with subPath, which is not updated at all.

immutable: true freezes an object: no updates, less API server and kubelet load, and the only way to change it is to create a new one and point the workload at it. This guide covers the three consumption forms, the update semantics that trip people up, Secret types and their expected keys, the ImagePullBackOff-class failures a missing key causes, and the practices that keep credentials out of kubectl describe output.

Basic Syntax & Core Signatures

yaml
apiVersion: v1 kind: ConfigMap metadata: name: app-config data: LOG_LEVEL: info app.yaml: | server: port: 8080 binaryData: logo.png: iVBORw0KGgoAAAANSUhEUg... # base64 immutable: true
yaml
apiVersion: v1 kind: Secret metadata: name: app-secret type: Opaque stringData: # plaintext here; merged into data by the API server DATABASE_URL: postgres://app:s3cret@db:5432/app data: # base64 in manifests and in `kubectl get -o yaml` API_KEY: c2VjcmV0LWtleQ== immutable: true
yaml
# Consumption: single keys as env env: - name: LOG_LEVEL valueFrom: configMapKeyRef: { name: app-config, key: LOG_LEVEL } - name: DATABASE_URL valueFrom: secretKeyRef: { name: app-secret, key: DATABASE_URL, optional: false } # All keys from both objects envFrom: - configMapRef: { name: app-config, optional: false } - secretRef: { name: app-secret } - configMapRef: { name: app-config-extra, prefix: EXTRA_ } # Mount as files: one file per key, whole object volumes: - name: config configMap: { name: app-config, defaultMode: 0444 } - name: secrets secret: { secretName: app-secret, defaultMode: 0400 } volumeMounts: - { name: config, mountPath: /etc/app, readOnly: true } # One key as one file, custom path, custom mode volumes: - name: tls secret: secretName: api-tls items: - key: tls.crt path: server.crt mode: 0400
bash
# Create without writing YAML kubectl create configmap app-config --from-file=./config --from-literal=LOG_LEVEL=info kubectl create secret generic app-secret --from-literal=DATABASE_URL='postgres://...' kubectl create secret docker-registry regcred --docker-server=ghcr.io \ --docker-username=bot --docker-password="$TOKEN" # Inspect (base64) and decode individual keys kubectl get secret app-secret -o jsonpath='{.data.API_KEY}' | base64 -d; echo kubectl describe secret app-secret # keys and sizes only, no values # After changing a ConfigMap, env vars do not follow: restart deliberately kubectl rollout restart deployment/api

Key Comparison Table

ConsumptionUpdates at runtimeFailure if key missingNotes
env.valueFromNoPod stuck CreateContainerConfigError unless optional: trueOne key
envFromNoPod starts without it if optional: trueInvalid keys skipped with an event
Volume mountYes, after ~1 minutePod stuck mounting, unless optional: trueOne file per key
Volume with subPathNoSameKnown limitation: not watched
Projected volumeYes, like a mountSameCombine ConfigMap, Secret, downward API, SA token
Secret typeRequired keysUsed for
OpaqueAnyGeneral credentials
kubernetes.io/tlstls.crt, tls.keyIngress TLS
kubernetes.io/dockerconfigjson.dockerconfigjsonimagePullSecrets
kubernetes.io/basic-authusername, passwordBasic auth conventions
kubernetes.io/ssh-authssh-privatekeyGit over SSH
kubernetes.io/service-account-token(managed)Long-lived SA tokens (legacy)
CommandEffect
kubectl get cmList ConfigMaps
kubectl describe cm app-configKeys and values in plaintext
kubectl get secret -o jsonpath='{.data.K}' | base64 -dDecode one value
kubectl create cm --from-file=dir/One key per file in the directory
kubectl get cm app-config -o yamlFull manifest with data

Practical Edge Cases & Deep Dives

1. CreateContainerConfigError: a missing key with no optional

bash
$ kubectl get pod api-1 NAME READY STATUS RESTARTS AGE api-1 0/1 CreateContainerConfigError 0 3m $ kubectl describe pod api-1 | grep -A2 'Error:' Error: couldn't find key DATABASE_URL in Secret default/app-secret

The pod is scheduled and the kubelet refuses to create the container, so there is nothing to exec into and no application log. Either the key is spelled differently (DATABASE_URL versus database_url), the Secret is in another namespace, or the Secret exists but envFrom and valueFrom disagree about which object holds it. optional: true converts this into a start-without-it situation, which is right for genuinely optional tuning values and wrong for required credentials — a pod starting without its database URL fails later and less clearly.

2. Changing a ConfigMap does not change a running pod's environment

bash
$ kubectl edit cm app-config # LOG_LEVEL: debug $ kubectl exec deploy/api -- printenv LOG_LEVEL info # unchanged

Environment variables are set when the container process is created. The API server's new value exists, the pod's spec still references the object, and nothing in between reconciles them. The options are a rollout (kubectl rollout restart, which recreates pods with the new environment) or a mounted file, which the kubelet does refresh. This is the main reason the "mount config as a file and reload on SIGHUP" pattern survives: a file-mounted ConfigMap updates within roughly a minute (kubelet sync period plus API cache propagation), and the application decides when to re-read it.

bash
kubectl get cm app-config -o jsonpath='{.data.LOG_LEVEL}' kubectl exec deploy/api -- cat /etc/app/LOG_LEVEL

3. subPath mounts never update, and the Helm trap

yaml
volumeMounts: - name: config mountPath: /etc/app/app.yaml subPath: app.yaml # mounts one file, and it will NOT be refreshed

subPath exists because mounting a whole directory over an existing one hides the image's contents (and mounting over /etc breaks DNS). But a subPath mount is resolved once at container start; the projected-volume update mechanism does not reach it. The trade-off is explicit: subPath for files that must sit at a precise path and rarely change (a TLS certificate that triggers a restart anyway), a plain mount for configuration that must reload.

A related trap: mounting a ConfigMap at /etc/nginx/conf.d hides whatever nginx's image already had there; the mount replaces the directory. Use items to control file names, or mount at a different path and include the files from the image's own config.

4. Secret values in the place you did not expect

bash
$ kubectl describe pod api-1 | grep -i env DATABASE_URL: <set to the key 'DATABASE_URL' in secret 'app-secret'> # safe $ kubectl get secret app-secret -o yaml # base64, readable by anyone with get secrets $ kubectl exec deploy/api -- cat /proc/1/environ | tr '\0' '\n' | grep DATABASE DATABASE_URL=postgres://app:s3cret@db:5432/app # plaintext in the process environment

describe protects secret values but not config values, and every process that receives a secret as an environment variable exposes it to ps e, /proc/<pid>/environ, crash dumps, and child processes that inherit it. A mounted file has narrower exposure (mode-controlled, memory-backed) but is still readable inside the container. The real protections are RBAC (get/list on Secrets is effectively read-all), encryption at rest in etcd, and not storing long-lived credentials in Kubernetes objects at all — external secret operators, sealed secrets, or workload identity are the durable answers.

5. envFrom and keys that are not valid variable names

bash
$ kubectl describe pod api-1 | grep -i event -A3 Warning InvalidVariableNames ... Skipped invalid env names: app.yaml, log-level

envFrom injects every key as a variable, and a key such as app.yaml or log-level is not a legal shell identifier, so it is silently skipped and an event is emitted. envFrom also does not let you rename keys; prefix only prepends. When the object mixes configuration files and variables, split it: one ConfigMap for variables consumed by envFrom, one for files consumed by a mount.

6. Immutability as an operational tool

bash
$ kubectl patch cm app-config -p '{"data":{"LOG_LEVEL":"debug"}}' The ConfigMap "app-config" is immutable: you may only edit labels and annotations

immutable: true prevents accidental edits to objects that pods depend on, and reduces the kubelet's watch load on clusters with thousands of them. The workflow becomes versioned: create app-config-v2, update the Deployment's reference, roll out, and delete the old one when no pod uses it. That matches how a reviewed configuration change should work, and it makes a rollback a one-line reference change instead of an in-place edit nobody recorded.

Common Runtime & Configuration Mistakes

SymptomCauseFix
CreateContainerConfigErrorMissing key without optionalCreate the key or the object; check the namespace
Pod starts without a required valueoptional: true used too broadlyRemove optional for required values
Changed ConfigMap has no effect on envEnv is fixed at container startkubectl rollout restart
Mounted file not updatedsubPath mountUse a whole-volume mount
Mounted file updates but app ignores itApp reads config onceReload on SIGHUP or watch the file
Environment variable missing with no errorInvalid key name in envFromRename the key; check InvalidVariableNames events
ImagePullBackOff on a private registryNo imagePullSecrets, or wrong Secret typekubectl create secret docker-registry, reference it
TLS Secret rejectedNot type kubernetes.io/tls, or keys misnamedtls.crt/tls.key, correct type
Secret appears garbledBase64 is not encryption; value double-encodedbase64 -d, or stringData on create
ConfigMap apply rejectedOver 1 MiBSplit it, or mount a real volume
Secret visible in kubectl get pods -o yaml?It is a reference, not a valueValues only appear with get secret
Cannot patch an object at allimmutable: trueCreate a new versioned object

ConfigMaps and Secrets are configuration delivery, not configuration management: they carry values to pods through a channel with precise update rules (env never, file eventually, subPath never), and their security properties come from the cluster around them — RBAC on Secrets, encryption at rest, and the discipline of keeping long-lived credentials out of etcd entirely.

Try matching browser tools

All tools

Small utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Kubernetes guides

More in Kubernetes

All Kubernetes guides

The guides filed next to this one in Kubernetes, in library order.