Notes from Testing SMB/CIFS CSI driver with OpenShift

1. Login to the OpenShift cluster oc login. You’ll need to do this with a password not kubeconfig.
2. Clone git clone https://github.com/prb112/openshift-samba
3. Change to cd openshift-samba
4. Create the Project oc new-project samba-test
5. Update Project permissions

oc label namespace/samba-test security.openshift.io/scc.podSecurityLabelSync=false --overwrite
oc label namespace/samba-test pod-security.kubernetes.io/enforce=privileged --overwrite
oc label namespace/samba-test pod-security.kubernetes.io/audit=privileged --overwrite
oc label namespace/samba-test pod-security.kubernetes.io/warn=privileged --overwrite

6. Enable incluster resolution

$ oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge

7. Run ./enable-registry-and-push.sh

$ ./enable-registry-and-push.sh
=== Image successfully pushed to OpenShift registry ===
You can now use this image in your deployments with: default-route-openshift-image-registry.apps.kt-test-cp4ba-1174.powervs-openshift-ipi.cis.ibm.net/samba-test/samba:latest

8. You can create the secret with oc create secret generic smbcreds --from-literal username=USERNAME --from-literal password="PASSWORD"
9. Setup setup the SMB server:

cat << EOF | oc apply -f -
---
kind: Service
apiVersion: v1
metadata:
  name: smb-server
  namespace: samba-test
  labels:
    app: smb-server
spec:
  type: ClusterIP
  selector:
    app: smb-server
  ports:
    - port: 445
      name: smb-server
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: smb-client-provisioner
  namespace: samba-test
---
kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: smb-server
  namespace: samba-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: smb-server
  template:
    metadata:
      name: smb-server
      labels:
        app: smb-server
    spec:
      serviceAccountName: smb-client-provisioner
      nodeSelector:
        kubernetes.io/os: linux
        kubernetes.io/hostname: worker-0
      containers:
        - name: smb-server
          image: image-registry.openshift-image-registry.svc:5000/samba-test/samba:latest
          ports:
            - containerPort: 445
          securityContext:
            privileged: true
            capabilities:
                add:
                - CAP_SYS_ADMIN
                - CAP_FOWNER
                - NET_ADMIN
                - SYS_ADMIN
                drop:
                - ALL
            runAsUser: 0
            runAsNonRoot: false
            readOnlyRootFilesystem: false
            allowPrivilegeEscalation: true
          volumeMounts:
            - mountPath: /export/smbshare
              name: data-volume
      volumes:
        - name: data-volume
          hostPath:
            path: /var/smb
            type: DirectoryOrCreate
EOF

10. Set the permissions oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:samba-test:smb-client-provisioner and oc adm policy add-scc-to-user privileged -z smb-client-provisioner -n samba-test
11. Kill the existing pods oc delete rs --all -n samba-test
12. Reset the samba-test permissions

oc rsh smb-server-0
chmod -R 777 /export

13. Check the connectivity works:

# oc rsh smb-server-0
$ smbclient //smb-server.samba-test.svc.cluster.local/data -U USERNAME --password=PASSWORD -W WORKGROUP
$ mkdir /export/abcd

14. Then you can create the SMB test using.

cat <<EOF | oc apply -f -
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb
provisioner: smb.csi.k8s.io
parameters:
  source: //smb-server.samba-test.svc.cluster.local/data
  csi.storage.k8s.io/provisioner-secret-name: smbcreds
  csi.storage.k8s.io/provisioner-secret-namespace: samba-test
  csi.storage.k8s.io/node-stage-secret-name: smbcreds
  csi.storage.k8s.io/node-stage-secret-namespace: samba-test
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1001
  - gid=1001
  - noperm
  - mfsymlinks
  - cache=strict
  - noserverino  # required to prevent data corruption
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-smb-1005
  namespace: samba-test
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: smb
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-smb
  namespace: samba-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-smb
  template:
    metadata:
      labels:
        app: nginx-smb
    spec:
      containers:
        - image: registry.access.redhat.com/ubi10/nginx-126@sha256:8e282961aa38ee1070b69209af21e4905c2ca27719502e7eaa55925c016ecb76
          name: nginx-smb
          command:
            - "/bin/sh"
            - "-c"
            - while true; do echo $(date) >> /mnt/outfile; sleep 1; done
          volumeMounts:
            - name: smb01
              mountPath: "/mnt"
              readOnly: false
      volumes:
        - name: smb01
          persistentVolumeClaim:
            claimName: pvc-smb-1005
EOF

15. Find the test pod oc get pod -l app=nginx-smb
16. Connect to the test pod and load a test file

# oc rsh pod/nginx-smb-6b55dc568-mbk9t
$ dd if=/dev/random of=/mnt/testfile bs=1M count=10
$ sha256sum /mnt/testfile
2bc558e0ccf2995a23cfa14c5cc500d9b4192b046796eb9fbfce772140470223  /mnt/testfile

17. Rollout restart

# oc rollout restart deployment nginx-smb

18. Find the test pod oc get pod -l app=nginx-smb
19. Connect to the test pod and load a test file

# oc rsh pod/nginx-smb-64cbbb9f56-7zfv7
$ sha256sum /mnt/testfile
2bc558e0ccf2995a23cfa14c5cc500d9b4192b046796eb9fbfce772140470223  /mnt/testfile

The sha256sum should agree with the first one.

20. Restart the SMB Server

oc rollout restart statefulset smb-server

21. Connect to the test pod and load a test file

# oc rsh pod/nginx-smb-64cbbb9f56-7zfv7
$ sha256sum /mnt/testfile
2bc558e0ccf2995a23cfa14c5cc500d9b4192b046796eb9fbfce772140470223  /mnt/testfile

These should all agree.

That’s all for testing. (I tried it out on yoru system.)