Category: IBM Power Systems

DRAFT This is not a complete article. I haven’t yet fully tested and vetted the steps I built. I will come back and hopefully update.

Kubernetes orchestrates Pods across multiple nodes. When a Pod lands on a node, the Kubelet admits the Pod and its containers, and manages the lifecycle of the containers. When the Pod is terminated, the kubelet sends a SIGTERM signal to the running processes. In Kubernetes Enhancement – Container Stop Signals #4960, custom Pod stopSignal is allowed: spec.containers[].lifecycle.stopSignal and you can use one of sixty-five additional stop signals to stop the Pod. While behind a feature gate, you can see supportedStopSignalsLinux.

For example, a user may use SIGQUIT signal to stop a container in the Pod. To do so with kind,

1. Enable the ContainerStopSignals featuregate in a kind config called kind-cluster-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
  ContainerStopSignals: true
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
        extraArgs:
          v: "1"
    scheduler:
        extraArgs:
          v: "1"
    controllerManager:
        extraArgs:
          v: "1"
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        v: "1"
- role: worker
  kubeadmConfigPatches:
  - |
    kind: JoinConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        v: "1"

2. Download kind

mkdir -p dev-cache
GOBIN=$(PWD)/dev-cache/ go install sigs.k8s.io/kind@v0.29.0

3. Start the kind cluster

KIND_EXPERIMENTAL_PROVIDER=podman dev-cache/kind create cluster \
		--image quay.io/powercloud/kind-node:v1.33.1 \
		--name test \
		--config kind-cluster-config.yaml\
		--wait 5m

4. Create a namespace

apiVersion: v1
kind: Namespace
metadata:
  labels:
    kubernetes.io/metadata.name: lifecycle-test
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: v1.24
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: v1.24
  name: lifecycle-test

5. Create a Pod

apiVersion: v1
kind: Pod
metadata:
  name: test
  namespace: lifecycle-test
spec:
  containers:
  - name: test
    command: ["/bin/sh", "-c"]
    args:
      - function cleanup() { echo "CALLED SIGQUIT"; };
        trap cleanup SIGQUIT;
        sleep infinity
    image: registry.access.redhat.com/ubi9/ubi
    lifecycle:
      stopSignal: SIGQUIT

6. Check kubectl describe pod/test -n lifecycle-test

You’ve seen how this feature functions with Kubernetes and can take advantage of ContainerStopSignals in your environment.

References

1. Tracker: Kubernetes Enhancement – Container Stop Signals #4960 issue 30051
2. KEP-4960: Container Stop Signals
3. Kubernetes Documentation: Container Lifecycle Hooks
4. An Introductory Guide to Managing the Kubernetes Pods Lifecycle
5. Stop Signals

Recently, I ran across Kubernetes Enhancement Proposal (KEP) 4265 where the authors update the Pod.spec.procMount capability to manage /proc visibility in a Pod’s security context. With this KEP moving to on-by default in v1.29.0, Unmasked disables masking and allows all paths in /proc (not just read-only).

What this means is the Default procMount prevents containers from accessing sensitive kernel data or interacting with host-level processes. With this enhancement, you can run unprivileged containers inside a container (a container-in-a-container), build container images within a Pod, and use buildah in a Pod.

The authors said it best in the KEP:

The /proc filesystem is a virtual interface to kernel data structures. By default, Kubernetes instructs container runtimes to mask or restrict access to certain paths within /proc to prevent accidental or malicious exposure of host information. But this becomes problematic when users want to:

Here is an example of creating a Pod:

1. create the project

oc new-project proc-mount-example

2. Create the Pod

cat << EOF | oc apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: nested-container-builder
  namespace: proc-mount-example
spec:
  securityContext:
    runAsUser: 0
  containers:
  - name: builder
    image: registry.access.redhat.com/ubi9/ubi
    securityContext:
      privileged: true
      procMount: Unmasked
    command: ["/bin/sh"]
    args: ["-c", "sleep 3600"]
EOF

3. Switch to terminal and install podman

oc rsh nested-container-builder
dnf install -y podman

4. Change the Shell (so you know when the parent is in focus…)

export PS1="parent-container# "
podman run --name abcd --rm -it registry.access.redhat.com/ubi9/ubi sh

5. Run a privileged command again

parent-container# podman run --name abcd --rm -it registry.access.redhat.com/ubi9/ubi sh
sh-5.1# dnf install -y podman

6. Now Run another in the nested one, you’ll see a failure in the /dev/net/tun.

sh-5.1# podman run --name abcd --rm -it registry.access.redhat.com/ubi9/ubi sh
Trying to pull registry.access.redhat.com/ubi9/ubi:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob ea2f7ff2baa2 done   | 
Copying config 4da9fa8b5a done   | 
Writing manifest to image destination
Storing signatures
ERRO[0018] Preparing container d402a22ebe452597a83b3795639f86e333c1dbb142703737d6d705c6a6f445c7: setting up Pasta: pasta failed with exit code 1:
                Failed to open() /dev/net/tun: No such file or directory
                                                                        Failed to set up tap device in namespace 
Error: mounting storage for container d402a22ebe452597a83b3795639f86e333c1dbb142703737d6d705c6a6f445c7: creating overlay mount to /var/lib/containers/storage/overlay/ab589890d52b88e51f1f945b55d07ac465de1cefd2411d8fab33b4d2769c4404/merged, mount_data="lowerdir=/var/lib/containers/storage/overlay/l/K6CXJGRTW32MPWEIMAH4IGCNZ5,upperdir=/var/lib/containers/storage/overlay/ab589890d52b88e51f1f945b55d07ac465de1cefd2411d8fab33b4d2769c4404/diff,workdir=/var/lib/containers/storage/overlay/ab589890d52b88e51f1f945b55d07ac465de1cefd2411d8fab33b4d2769c4404/work,nodev,volatile": using mount program /usr/bin/fuse-overlayfs: unknown argument ignored: lazytime
fuse: device not found, try 'modprobe fuse' first
fuse-overlayfs: cannot mount: No such file or directory
: exit status 1

It has the default access:

• Default: Maintains the current behavior—masking sensitive /proc paths. If procMount is not specified, it defaults to Default, ensuring backward compatibility and preserving security for most workloads.
• Unmasked: Bypasses the default masking, giving the container full access to /proc.

Allowing unmasked access to /proc is a privileged operation. A container with root access and an unmasked /proc could potentially interact with the host system in dangerous ways. This powerful feature should be carefully used.

Good luck.

References

• Kubernetes Enhancement Proposal (KEP) 4265
• Enhancement Tracker: PR add ProcMount option on by default beta
• Configure a Security Context for a Pod or Container: Managing access to the /proc filesystem
• jessfraz: Building Container Images Securely on Kubernetes