Kubernetes Exit Code 137 OOMKilled vs Exit Code 134 Heap Limits
137 is SIGKILL from the cgroup or a liveness probe; 134 is an abort, usually V8's own heap ceiling. Telling them apart and sizing the two limits together.

Introduction & Quick Summary
Symptom. A pod restarts every few minutes. kubectl get pod shows OOMKilled or Error, and kubectl describe pod prints Last State: Terminated Reason: OOMKilled Exit Code: 137 — or the same pod shows Exit Code: 134 with FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory in the previous container's log. Memory graphs look fine right up to the restart, and the app logged nothing before dying.
Cause. Both are memory failures, but at different layers. 137 is 128 + 9 (SIGKILL): the kernel's cgroup OOM killer chose the process because the container's working set exceeded its memory.limit_in_bytes, or the kubelet killed it because a liveness probe failed — both send SIGKILL, and only the former sets Reason: OOMKilled. 134 is 128 + 6 (SIGABRT): the process aborted itself, in Node typically because V8's old-space limit was reached while the container still had room, or a native library called abort().
Fix. Make the runtime's internal ceiling sit below the container's ceiling. For Node, set NODE_OPTIONS=--max-old-space-size to roughly 70–80% of the memory limit and leave the rest for buffers, threads, and the native stack. Set requests == limits so the pod is Guaranteed and is not evicted for a neighbour's pressure, add a startupProbe so a slow boot is not killed by liveness, and watch container_memory_working_set_bytes rather than the application's own heap metric — the gap between them is the leak you cannot see from inside.
Core Mechanics, Signatures & Reproducible Setup
Reproduce the 134 case in a pod with a limit far above the heap, which is exactly how it happens in production:
apiVersion: v1
kind: Pod
metadata:
name: node-134
spec:
containers:
- name: app
image: node:22-alpine
command: ["node", "-e", "const a=[];for(;;)a.push(Buffer.alloc(1024*1024).toString('hex'))"]
resources:
requests: { memory: "256Mi", cpu: "100m" }
limits: { memory: "2Gi", cpu: "1" }The container limit is 2 GiB and V8's default old space is derived from what Node believes is available, so the process aborts with 134 when V8 gives up, long before the cgroup would kill it. Set the ceiling explicitly:
env:
- name: NODE_OPTIONS
value: "--max-old-space-size=1200" # ~60% of 2Gi, headroom for buffers
resources:
requests: { memory: "2Gi", cpu: "500m" }
limits: { memory: "2Gi", cpu: "1" }Read the two states apart with the evidence Kubernetes keeps:
kubectl get pod app-7d9 -o jsonpath='{.status.containerStatuses[0].lastState.terminated}'
kubectl describe pod app-7d9 | sed -n '/Last State/,/Ready/p'
kubectl logs app-7d9 --previous --tail=100
kubectl get events --field-selector involvedObject.name=app-7d9 --sort-by=.lastTimestamplastState.terminated.reason is OOMKilled for a cgroup kill, Error for a non-zero exit, and Completed for 0. The Events section distinguishes a probe kill: Killing container ... Container app failed liveness probe, will be restarted and Liveness probe failed: ... appear next to Killing with no OOMKilled reason.
Key Comparison Table
| Exit code | Signal | Who kills it | Where the evidence is |
|---|---|---|---|
| 137 | 9 SIGKILL | cgroup OOM killer | Reason: OOMKilled, node dmesg |
| 137 (no OOMKilled) | 9 SIGKILL | kubelet, failed liveness/startup probe | Events: failed liveness probe |
| 134 | 6 SIGABRT | The process (V8 fatal, glibc abort) | Previous log: FATAL ERROR |
| 143 | 15 SIGTERM | kubelet, graceful stop, node drain | terminationGracePeriodSeconds |
| 139 | 11 SIGSEGV | Native crash, bad pointer | Stack trace in log or core |
| 1 | — | Application error | Application log |
| Investigation | Command | Tells you |
|---|---|---|
| Reason and code | kubectl get pod -o jsonpath='…lastState.terminated' | OOMKilled vs Error |
| Probe kills | kubectl get events --sort-by=.lastTimestamp | Liveness, startup, readiness |
| Last words | kubectl logs --previous | V8 fatal error text |
| Live memory | kubectl top pod, cAdvisor | Working set vs limit |
| Node-level OOM | dmesg -T | grep -i oom on the node | Which container, which size |
| Heap inside the app | /debug/pprof, process.memoryUsage() | Heap vs RSS gap |
Practical Edge Cases, Error Fixes & Deep Dive Code Examples
1. OOMKilled with a memory graph that never reached the limit
container_memory_working_set_bytes is sampled, and the last sample before a kill can be one scrape interval old; a burst between samples is invisible. Worse, the metric the app exports (process.memoryUsage().heapUsed) excludes Buffers, ArrayBuffer, native allocations, worker threads, and the V8 code space. Log the difference:
setInterval(() => {
const m = process.memoryUsage();
log.info({ rss: m.rss, heap: m.heapUsed, external: m.external, arrayBuffers: m.arrayBuffers });
}, 30_000);rss growing while heap is flat means off-heap growth: leaked Buffers, unclosed streams, or a native addon. Raising --max-old-space-size will not fix it.
2. Exit 137 from a liveness probe, not memory
A probe with initialDelaySeconds: 5 on a service that takes 40 s to warm up kills it forever, and each restart begins the warm-up again. Use a startup probe with a generous budget and let liveness only cover the steady state:
startupProbe:
httpGet: { path: /healthz, port: 3000 }
failureThreshold: 30
periodSeconds: 2
livenessProbe:
httpGet: { path: /healthz, port: 3000 }
periodSeconds: 10
failureThreshold: 33. Heap set above the container limit
--max-old-space-size=2048 in a 512Mi container is the mirror image of 134: V8 never aborts, the cgroup kills at 512Mi, and you get 137 with no diagnostic from the app. Keep the ratio explicit and assert it at boot:
const limitMb = Number(process.env.MEMORY_LIMIT_MB ?? 0);
const heapMb = Number(process.execArgv.find((a) => a.startsWith('--max-old-space-size='))?.split('=')[1] ?? 0);
if (limitMb && heapMb > limitMb * 0.85) log.warn({ limitMb, heapMb }, 'heap too close to cgroup limit');4. Worker threads and cluster forks multiply the ceiling
--max-old-space-size applies per isolate. Four workers in a 1 GiB container each get the flag and compete for the same cgroup: total heap can reach four times the number you set. Divide deliberately:
import os from 'node:os';
const workers = Number(process.env.WORKERS ?? 2);
const perWorkerMb = Math.floor(totalMb / workers * 0.7);5. JVM and .NET have the same split
-XX:MaxRAMPercentage=75 is the JVM equivalent of --max-old-space-size, and without it older JDKs read the host's memory rather than the cgroup limit — the same 137/134 confusion appears as java.lang.OutOfMemoryError: Java heap space (abort) beside OOMKilled (kernel). Both runtimes need the container limit discovered at startup, which cgroup v2 exposes in memory.max.
6. Capturing a heap snapshot before the kill
Once the OOM killer acts, there is nothing to inspect. Run with a headroom-triggered snapshot, or attach an ephemeral container with kubectl debug and take a profile while it is alive:
kubectl debug -it app-7d9 --image=node:22-alpine --target=app -- sh
node -e "process.kill(1, 'SIGUSR2')" # if your app writes a snapshot on SIGUSR2--previous logs plus one heap snapshot taken at 80% of the limit identify more leaks than any dashboard.
Common Compilation/Runtime Mistakes & Version Gotchas
limitswithoutrequests. The pod becomesBurstable; the scheduler packs it onto a busy node, and the kernel reclaims its memory under pressure before the cgroup limit is hit.- Relying on
Reason: OOMKilledonly. A container killed by the kernel while another container on the node was over its limit can be reported without that reason; correlate with node events. - cgroup v2 accounting differs. Page cache counts toward the memory limit differently, and a process reading large files can be killed while its anonymous memory is small.
--memory-swapinteracts as well. - Node 20+ reads the cgroup limit correctly, but only for the main thread's heap. Buffers,
SharedArrayBuffer, and native memory are outside it; the 70–80% ratio exists for exactly that reason. terminationGracePeriodSeconds: 30with slow drains. The pod receives SIGTERM (143) and is SIGKILLed (137) at the deadline, which shows as 137 without OOMKilled — raise the grace period or shut down faster.restartPolicy: Alwayshides the pattern. Count restarts (kubectl get pod -o jsonpath='{.status.containerStatuses[0].restartCount}') and alert on the rate; a crash loop is a trend, not an event.resources.limits.memorybelow ~64Mi. Node cannot even start; the failure looks like a missing binary or an instant OOM with no log.
Related tools. Decode the previous container's structured log lines with the JSON formatter, convert the restart timestamps into a comparable timeline with the timestamp converter, and encode the memory-limit values you pass through Helm or envsubst with the base64 encoder when they arrive from a secret.
Try matching browser tools
All toolsSmall utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Kubernetes guides
All Kubernetes guidesCrashLoopBackOff: Reading Exit Codes and Debugging Restart Loops
The backoff is only the waiting state. The evidence is in Last State, the exit code, and the previous container's last log line, and each points to a different fix.
ImagePullBackOff and ErrImagePull: Reading the Real Pull Error
ImagePullBackOff hides five different failures. The event text names which one: wrong reference, missing credentials, wrong architecture, unreachable registry, or a rate limit.
kubectl Output: jsonpath, custom-columns, and the Debugging Commands Worth Knowing
jsonpath for one value in a script, custom-columns for tables people read, and kubectl debug when the container has no shell. Includes the three quoting layers that break jsonpath.
Kubernetes Namespaces, ResourceQuotas, and LimitRanges
A namespace is only a boundary when quota and limit range make it one. Aggregate caps, per-pod defaults, the admission errors they produce, and namespaces stuck Terminating.
More in Kubernetes
All Kubernetes guidesThe guides filed next to this one in Kubernetes, in library order.