Diagnosing a Pod Crash on OpenShift: SIGSEGV vs OOMKill

When a your pod crashes with a segmentation fault signal near address 0x0 or 0x20, that points to an invalid pointer dereference — not an out-of-memory kill. The two failure modes look similar on the surface but require different investigation paths.

  1. Confirm the Actual Exit Cause
oc get pod <pod> -n <namespace> -o json \
  | jq '.status.containerStatuses[]'
SignalExit CodeMeaning
SIGSEGV139 / signal 11Segmentation fault → investigate the crash
—137 / OOMKilledOut of memory → review resource limits instead

If you see OOMKilled, stop here and adjust the pod’s memory limits. If you see exit code 139, continue below.

  1. Open a Debug Shell on the Affected Node
oc debug node/<node>

Inside the debug shell:

chroot /host
coredumpctl list
coredumpctl info <host-pid>

Match the entry by crash timestamp and executable name. Use the PID reported by coredumpctl — not the container’s PID 1.

  1. Export the Core Dump

Still inside the debug shell, write the dump to a location with sufficient free space:

umask 077
coredumpctl --output=/var/tmp/app.core dump <host-pid>
  1. Copy the Dump Locally

From a second terminal (keep the debug session open), copy the file using the debug pod’s name:

oc cp -n <debug-namespace> \
  <debug-pod>:/host/var/tmp/app.core ./app.core

Next Steps

Hand the app.core file off to the engineering team for analysis. A kernel dump is unlikely to add value here — the core dump from the process itself is what matters for a near-zero pointer fault.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *