Tag: openshift

Here are my notes for setting up the SIG’s nfs-provisioner. You should follow these directions to setup the nfs-provisioner kubernetes-sigs/nfs-subdir-external-provisioner.

1. If you haven’t already, you need to create the nfs-provisioner namespace.

a. Create the namespace

oc new-project nfs-provisioner

b. Annotate the namespace with elevated privileges so we can create NFS mounts

# oc label namespace/nfs-provisioner security.openshift.io/scc.podSecurityLabelSync=false --overwrite=true
namespace/nfs-provisioner labeled

# oc label namespace/nfs-provisioner pod-security.kubernetes.io/enforce=privileged --overwrite=true
namespace/nfs-provisioner labeled

# oc label namespace/nfs-provisioner pod-security.kubernetes.io/enforce-version=v1.24 --overwrite=true
namespace/nfs-provisioner labeled

# oc label namespace/nfs-provisioner pod- security.kubernetes.io/audit=privileged --overwrite=true
namespace/nfs-provisioner labeled

# oc label namespace/nfs-provisioner pod-security.kubernetes.io/warn=privileged --overwrite=true
namespace/nfs-provisioner labeled

1. Download the storage-class-nfs-template

# curl -O -L https://github.com/IBM/ocp4-power-workload-tools/manifests/storage/storage-class-nfs-template.yaml

1. Setup Authorization

oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:nfs-provisioner:nfs-client-provisioner

• Process the template with the NFS_PATH and NFS_SERVER

# oc process -f storage-class-nfs-template.yaml -p NFS_PATH=/data -p NFS_SERVER=10.17.2.138 | oc apply -f –

deployment.apps/nfs-client-provisioner created
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
storageclass.storage.k8s.io/nfs-client created

1. Get the pods

oc get pods

NAME                                     READY   STATUS    RESTARTS   AGE

nfs-client-provisioner-b8764c6bb-mjnq9   1/1     Running   0          36s

1. Check the storage class… You should see the nfs-client listed. This is the default.

❯ oc get sc

NAME         PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE

nfs-client   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  3m27s

If you see more than the nfs-client listed, you may have to change the defaults.

oc patch storageclass storageclass-name -p ‘{“metadata”: {“annotations”: {“storageclass.kubernetes.io/is-default-class”: “false”}}}’

Shows a Microservices Application running on Red Hat OpenShift Control Plane on IBM Power Systems with an Intel Worke

Here are some great updates for the first half of April 2024.

Sizing and configuring an LPAR for AI workloads

Sebastian Lehrig has a great introduction into CPU/AI/NUMA on Power10.

https://community.ibm.com/community/user/powerdeveloper/blogs/sebastian-lehrig/2024/03/26/sizing-for-ai

FYI: a new article is published – Improving the User Experience for Multi-Architecture Compute on IBM Power

More and more IBM® Power® clients are modernizing securely with lower risk and faster time to value with cloud-native microservices on Red Hat® OpenShift® running alongside their existing banking and industry applications on AIX, IBM i, and Linux. With the availability of Red Hat OpenShift 4.15 on March 19th, Red Hat and IBM introduced a long-awaited innovation called Multi-Architecture Compute that enables clients to mix Power and x86 worker nodes in a single Red Hat OpenShift cluster. With the release of Red Hat OpenShift 4.15, clients can now run the control plane for a Multi-Architecture Compute cluster natively on Power.

Some tips for setting up a Multi-Arch Compute Cluster

Setting up a multi-arch compute cluster manually, not using automation, you’ll want to follow this process:

1. Setup the Initial Cluster with the multi payload on Intel or Power for the Control Plane.
2. Open the network ports between the two environments

ICMP/TCP/UDP flowing in both directions

3. Configure the Cluster

a. Change any MTU between the networks

oc patch Network.operator.openshift.io cluster --type=merge --patch \
    '{"spec": { "migration": { "mtu": { "network": { "from": 1400, "to": 1350 } , "machine": { "to" : 9100} } } } }'

b. Limit CSI drivers to a single Arch

oc annotate --kubeconfig /root/.kube/config ns openshift-cluster-csi-drivers \
  scheduler.alpha.kubernetes.io/node-selector=kubernetes.io/arch=amd64

c. Disable offloading (I do this in the ignition)

d. Move the imagepruner jobs to the architecture that makes the most sense

oc patch imagepruner/cluster -p '{ "spec" : {"nodeSelector": {"kubernetes.io/arch" : "amd64"}}}' --type merge

e. Move the ingress operator pods to the arch that makes the most sense. If you want the ingress pods to be on Intel then patch the clsuter.

oc edit IngressController default -n openshift-ingress-operator

Change ingresscontroller.spec.nodePlacement.nodeSelector to use the kubernetes.io/arch: amd64 to move the workfload to Intel only.

f. use routing via host

oc patch network.operator/cluster --type merge -p \
  '{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"gatewayConfig":{"routingViaHost":true}}}}}'

Wait until the MCP is finished updating and has the latest MTU

g. Download the igntion file and host on the local network via http.

4. Create a new VSI worker and point to the ignition in userdata

{
    "ignition": {
        "version": "3.4.0",
        "config": {
            "merge": [
                {
                    "source": "http://${ignition_ip}:8080/ignition/worker.ign"
                }
            ]
        }
    },
    "storage": {
        "files": [
            {
                "group": {},
                "path": "/etc/hostname",
                "user": {},
                "contents": {
                    "source": "data:text/plain;base64,${name}",
                    "verification": {}
                },
                "mode": 420
            },
            {
                "group": {},
                "path": "/etc/NetworkManager/dispatcher.d/20-ethtool",
                "user": {},
                "contents": {
                    "source": "data:text/plain;base64,aWYgWyAiJDEiID0gImVudjIiIF0gJiYgWyAiJDIiID0gInVwIiBdCnRoZW4KICBlY2hvICJUdXJuaW5nIG9mZiB0eC1jaGVja3N1bW1pbmciCiAgL3NiaW4vZXRodG9vbCAtLW9mZmxvYWQgZW52MiB0eC1jaGVja3N1bW1pbmcgb2ZmCmVsc2UgCiAgZWNobyAibm90IHJ1bm5pbmcgdHgtY2hlY2tzdW1taW5nIG9mZiIKZmkKaWYgc3lzdGVtY3RsIGlzLWZhaWxlZCBOZXR3b3JrTWFuYWdlci13YWl0LW9ubGluZQp0aGVuCnN5c3RlbWN0bCByZXN0YXJ0IE5ldHdvcmtNYW5hZ2VyLXdhaXQtb25saW5lCmZpCg==",
                    "verification": {}
                },
                "mode": 420
            }
        ]
    }
}

${name} is base64 encoded.

5. Post configuration tasks

a. Configure shared storage using the nfs provisioner and limit to running from the architecture that is hosting the NFS shared volumes.

b. Approve the CSRs for the workers. Do this carefully as it’s possible to lose the count as it may include Machine updates/csrs.

6. Check the cluster operators and nodes it should be up and working.

The IBM team Bringing OpenShift Container Platform Multiple-Architecture Compute to IBM Power with sme.up posted an update on one of our OpenShift on Power customers who used Multi-Architecture Compute.

IBM announced the availability of Red Hat OpenShift 4.15 available on IBM Power. Read more about it in
https://community.ibm.com/community/user/powerdeveloper/blogs/brandon-pederson1/2024/03/15/red-hat-openshift-415-now-available-on-ibm-power

I worked on the following:

Red Hat OpenShift 4.14, Multi-Architecture Compute was introduced for the IBM Power and IBM Z platforms, enabling a single heterogeneous cluster across different compute architectures. With the release of Red Hat OpenShift 4.15, clients can now add x86 compute nodes to a multi-architecture enabled cluster running on Power. This simplifies deployment across different environments even further and provides a more consistent management experience. Clients are accelerating their modernization journeys with multi-architecture compute and Red Hat OpenShift by exploiting the best-fit architecture for different solutions and reducing cost and complexity of workloads that require multiple compute architectures.

FYI: Webinar: Getting started with Multi-Arch Compute workloads with your Red Hat OpenShift cluster

Summary



The Red Hat OpenShift Container Platform runs on IBM Power systems, offering a secure and reliable foundation for modernizing applications and running containerized workloads.

Multi-Arch Compute for OpenShift Container Platform lets you use a pair of compute architectures such as, ppc64le and amd64, within a single cluster. This exciting feature opens new possibilities for versatility and optimization for composite solutions that span multiple architectures.

Join Paul Bastide,  IBM Senior Software Engineer, as he introduces the background behind Multi-Arch Compute and then gets you started setting up, configuring, and scheduling workloads. After, Paul will take you through a brief demonstration showing common problems and solutions for running multiple architectures in the same cluster.

This presentation sets the background and gets you started so you can set up, configure, and scheduling workloads. There will be a brief demonstration showing common problems and solutions for running multiple architectures in the same cluster.

Please join me on 11 April 2024, 9:00 AM ET. Please share any questions by clicking on the Reply button. If you have not done so already, register here and download it to your calendar.