Category: IBM Power Systems
-
Weekly Notes
Here are my weekly learnings and notes:
Podman Desktop updates v1.0.1
Podman Desktop is an open source graphical tool enabling you to seamlessly work with containers and Kubernetes from your local environment.
In a cool update, the Podman Desktop team added support for OpenShift Local in v1.0.1 and Kind clusters are already there. We can do some advanced stuff. You may have to download extensions and upgrade Podman to v4.5.0.
❯ brew upgrade podman-desktop ... 🍺 podman-desktop was successfully upgraded!Skupper… interesting
Skupper is a layer 7 service interconnect. It enables secure communication across Kubernetes clusters with no VPNs or special firewall rules.
There is a new layer-7 interconnect. There is a sample
Red Hat OpenShift Container Platform 4.13.0 is generally available
I’ve been working on the product for 4.13.0 – oc new-app and new-build support.
Podman Cheat Sheet
Podman Cheat Sheet covers all the basic commands for managing images, containers, and container resources. Super helpful for those stuck finding the right command to build/manage or run your container.
File Integrity Operator: Using File Integrity Operator to support file integrity checks on OpenShift Container Platform on Power
My colleague has published a blog on File Integrity Operator.
As part of this series, I have written a blog on PCI-DSS and the Compliance Operator to have a secure and compliant cluster. Part of the cluster’s security and compliance depends on the File Integrity Operator – an operator that uses intrusion detection rules to verify the integrity of files and directories on cluster’s nodes.
https://community.ibm.com/community/user/powerdeveloper/blogs/aditi-jadhav/2023/05/24/using-file-integrity-operator-to-support-file-inte -
Weekly Notes
Here are my notes from the week:
- Subnet to CIDR block Cheat Sheet
- OpenShift Installer Provisioned Infrastructure for IBM Cloud VPC
rfc1878: Subnet CIDR Cheat Sheet
I found a great cheat sheet for CIDR subnet masks.
Mask value: # of Hex CIDR Decimal addresses Classfull 80.00.00.00 /1 128.0.0.0 2048 M 128 A C0.00.00.00 /2 192.0.0.0 1024 M 64 A E0.00.00.00 /3 224.0.0.0 512 M 32 A F0.00.00.00 /4 240.0.0.0 256 M 16 A F8.00.00.00 /5 248.0.0.0 128 M 8 A FC.00.00.00 /6 252.0.0.0 64 M 4 A FE.00.00.00 /7 254.0.0.0 32 M 2 A FF.00.00.00 /8 255.0.0.0 16 M 1 A FF.80.00.00 /9 255.128.0.0 8 M 128 B FF.C0.00.00 /10 255.192.0.0 4 M 64 B FF.E0.00.00 /11 255.224.0.0 2 M 32 B FF.F0.00.00 /12 255.240.0.0 1024 K 16 B FF.F8.00.00 /13 255.248.0.0 512 K 8 B FF.FC.00.00 /14 255.252.0.0 256 K 4 B FF.FE.00.00 /15 255.254.0.0 128 K 2 B FF.FF.00.00 /16 255.255.0.0 64 K 1 B FF.FF.80.00 /17 255.255.128.0 32 K 128 C FF.FF.C0.00 /18 255.255.192.0 16 K 64 C FF.FF.E0.00 /19 255.255.224.0 8 K 32 C FF.FF.F0.00 /20 255.255.240.0 4 K 16 C FF.FF.F8.00 /21 255.255.248.0 2 K 8 C FF.FF.FC.00 /22 255.255.252.0 1 K 4 C FF.FF.FE.00 /23 255.255.254.0 512 2 C FF.FF.FF.00 /24 255.255.255.0 256 1 C FF.FF.FF.80 /25 255.255.255.128 128 1/2 C FF.FF.FF.C0 /26 255.255.255.192 64 1/4 C FF.FF.FF.E0 /27 255.255.255.224 32 1/8 C FF.FF.FF.F0 /28 255.255.255.240 16 1/16 C FF.FF.FF.F8 /29 255.255.255.248 8 1/32 C FF.FF.FF.FC /30 255.255.255.252 4 1/64 C FF.FF.FF.FE /31 255.255.255.254 2 1/128 C FF.FF.FF.FF /32 255.255.255.255 1Thanks to the following sites for the clue to the rfc and the rfc.
Mutating WebHook to add Node Selectors
Thanks to these sites
- hmcts/k8s-env-injector provided inspiration for this approach and updates the code patterns for the latest kubernetes versions.
- phenixblue/imageswap-webhook provided the python based pattern for this approach.
- Kubernetes: MutatingAdmissionWebhook
I added some code to add annotations and nodeSelectors https://github.com/prb112/openshift-demo/tree/main/mutating
Installing OpenShift install provisioned infrastructure on IBM Cloud VPC
This document outlines installing the IPI IBMCloud using the
openshift-installer.As of OpenShift 4.13, you can install a cluster into an existing Virtual Private Cloud (VPC) on IBM Cloud VPC. The installation program provisions the required infrastructure, which you can then further customize.
This document describes the creation of OCP cluster using IPI (Installer Provisioned Infrastructure) on exiting IBM Cloud VPC.
This setup is used with the day-2 operations on PowerVS to make a multiarch compute cluster.
- Create IBM API Key
- Create the IAM Services
- Pick your build
- Deploy
1. Create IBM API Key
- Navigate to
API keysiam – api keys - Click
Create - Enter name
rdr-demo - Click
Create - Copy your API key, it’ll be used later on.
2. Create the IAM Services
- Navigate to
Service Idsiam – serviceids - click create service id with name
rdr-demoto identify your team. - assign access
Internet Services All Viewer, Operator, Editor, Reader, Writer, Manager, Administrator -- Cloud Object Storage All Viewer, Operator, Editor, Reader, Writer, Manager, Content Reader, Object Reader, Object Writer, Administrator -- IAM Identity Service All Viewer, Operator, Editor, Administrator, ccoctlPolicy, policycreate -- Resource group only ocp-dev-resource-group resource group Viewer, Administrator, Editor, Operator -- VPC Infrastructure Services All Viewer, Operator, Editor, Reader, Writer, Administrator, Manager3. Pick your build
I used 4.13.0-rc.7.
4. Deploy
- Connect to your jumpserver or bastion where you are doing the deployment.
Tip: it’s worth having
tmuxinstalled for this install (it’ll take about 1h30m)- Export the API KEY you created above
❯ export IC_API_KEY=<REDACTED>- Create a working folder
❯ mkdir -p ipi-vpc-414-rc7 ❯ cd ipi-vpc-414-rc7- Download the installers and extract to the binary folder.
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/ccoctl-linux.tar.gz ❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-client-linux.tar.gz ❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-install-linux.tar.gz ❯ tar xvf ccoctl-linux.tar.gz --dir /usr/local/bin/ ❯ tar xvf openshift-client-linux.tar.gz --dir /usr/local/bin/ ❯ tar xvf openshift-install-linux.tar.gz --dir /usr/local/bin/- Verify the openshift-install version is correct.
❯ openshift-install version openshift-install 4.13.0-rc.7 built from commit 3e0b2a2ec26d9ffcca34b361896418499ad9d603 release image quay.io/openshift-release-dev/ocp-release@sha256:aae5131ec824c301c11d0bf11d81b3996a222be8b49ce4716e9d464229a2f92b release architecture amd64- Copy over your pull-secret.
a. Login with your Red Hat id
b. Navigate to https://console.redhat.com/openshift/install/ibm-cloud
c. Scroll down the page and copy the pull-secret.
This pull-secret should work for you and save for later as
pull-secret.txtin the working directory.- Extract the CloudControlsRequest objects and create the credentials.
RELEASE_IMAGE=$(openshift-install version | awk '/release image/ {print $3}') oc adm release extract --cloud=ibmcloud --credentials-requests $RELEASE_IMAGE --to=rdr-demo ccoctl ibmcloud create-service-id --credentials-requests-dir rdr-demo --output-dir rdr-demo-out --name rdr-demo --resource-group-name ocp-dev-resource-group- Create the install-config
❯ openshift-install create install-config --dir rc7_2 ? SSH Public Key /root/.ssh/id_rsa.pub ? Platform ibmcloud ? Region jp-osa ? Base Domain ocp-multiarch.xyz (rdr-multi-is) ? Cluster Name rdr-multi-pb ? Pull Secret [? for help] ******************************************************************************** *********************************** INFO Manifests created in: rc7_1/manifests and rc7_1/openshift- Edit the install-config.yaml to add
resourceGroupName
platform: ibmcloud: region: jp-osa resourceGroupName: my-resource-group- Copy the generated ccoctl manifests over.
❯ cp rdr-demo-out/manifests/* rc7_1/manifests/- Create the manifests.
❯ openshift-install create manifests --dir=rc7_1 INFO Consuming OpenShift Install (Manifests) from target directory INFO Manifests created in: rc7_1/manifests and rc7_1/openshift- Create the cluster.
❯ openshift-install create cluster --dir=rc7_3 INFO Consuming Worker Machines from target directory INFO Consuming Common Manifests from target directory INFO Consuming Openshift Manifests from target directory INFO Consuming OpenShift Install (Manifests) from target directory INFO Consuming Master Machines from target directoryINFO Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.13-9.2/builds/413.92.202305021736-0/x86_64/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2.gz?sha256=222abce547c1bbf32723676f4977a3721c8a3788f0b7b6b3496b79999e8c60b3' INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2. Reusing... INFO Creating infrastructure resources... INFO Waiting up to 20m0s (until 12:09PM) for the Kubernetes API at https://api.xyz.ocp-multiarch.xyz:6443... INFO API v1.26.3+b404935 up INFO Waiting up to 30m0s (until 12:19PM) for bootstrapping to complete... INFO Destroying the bootstrap resources... INFO Waiting up to 40m0s (until 12:41PM) for the cluster at https://api.xyz.ocp-multiarch.xyz:6443 to initialize... INFO Checking to see if there is a route at openshift-console/console... INFO Install complete! INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/ipi-vpc-414-rc7/rc7_3/auth/kubeconfig' INFO Access the OpenShift web-console here: INFO Login to the console with user: "kubeadmin", and password: "xxxxxxxxx-wwwwww-xxxx-aas" INFO Time elapsed: 1h28m9s- Verify the cluster
a. set kubeconfig provided by installation
export KUBECONFIG=$(pwd)/rc7_1/auth/kubeconfigb. Check the nodes are Ready
❯ oc get nodes NAME STATUS ROLES AGE VERSION rdr-multi-ca-rc6-tplwd-master-0 Ready control-plane,master 5h13m v1.26.3+b404935 rdr-multi-ca-rc6-tplwd-master-1 Ready control-plane,master 5h13m v1.26.3+b404935 rdr-multi-ca-rc6-tplwd-master-2 Ready control-plane,master 5h13m v1.26.3+b404935 rdr-multi-ca-rc6-tplwd-worker-1-pfqjx Ready worker 4h47m v1.26.3+b404935 rdr-multi-ca-rc6-tplwd-worker-1-th8j4 Ready worker 4h47m v1.26.3+b404935 rdr-multi-ca-rc6-tplwd-worker-1-xl75m Ready worker 4h53m v1.26.3+b404935c. Check Cluster Operators
❯ oc get co NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE authentication 4.13.0-rc.6 True False False 4h43m baremetal 4.13.0-rc.6 True False False 5h5m cloud-controller-manager 4.13.0-rc.6 True False False 5h13m cloud-credential 4.13.0-rc.6 True False False 5h18m cluster-autoscaler 4.13.0-rc.6 True False False 5h5m config-operator 4.13.0-rc.6 True False False 5h7m console 4.13.0-rc.6 True False False 4h47m control-plane-machine-set 4.13.0-rc.6 True False False 5h5m csi-snapshot-controller 4.13.0-rc.6 True False False 4h54m dns 4.13.0-rc.6 True False False 4h54m etcd 4.13.0-rc.6 True False False 4h57m image-registry 4.13.0-rc.6 True False False 4h50m ingress 4.13.0-rc.6 True False False 4h51m insights 4.13.0-rc.6 True False False 5h kube-apiserver 4.13.0-rc.6 True False False 4h53m kube-controller-manager 4.13.0-rc.6 True False False 4h53m kube-scheduler 4.13.0-rc.6 True False False 4h52m kube-storage-version-migrator 4.13.0-rc.6 True False False 4h54m machine-api 4.13.0-rc.6 True False False 4h48m machine-approver 4.13.0-rc.6 True False False 5h5m machine-config 4.13.0-rc.6 True False False 5h6m marketplace 4.13.0-rc.6 True False False 5h5m monitoring 4.13.0-rc.6 True False False 4h45m network 4.13.0-rc.6 True False False 5h8m node-tuning 4.13.0-rc.6 True False False 4h54m openshift-apiserver 4.13.0-rc.6 True False False 4h47m openshift-controller-manager 4.13.0-rc.6 True False False 4h54m openshift-samples 4.13.0-rc.6 True False False 4h50m operator-lifecycle-manager 4.13.0-rc.6 True False False 5h6m operator-lifecycle-manager-catalog 4.13.0-rc.6 True False False 5h6m operator-lifecycle-manager-packageserver 4.13.0-rc.6 True False False 4h51m service-ca 4.13.0-rc.6 True False False 5h7m storage 4.13.0-rc.6 True False False 4h51mNote – Confirm that all master/worker nodes and operators are running healthy and true.
- Verify the browser login
A. Open Browser and Login to Console URL using available credentials. e.g.,
URL - https://console-openshift-console.apps.xxxxxx.ocp-multiarch.xyz Username – kubeadmin Password - <Generated Password>- destroy cluster Fire below mentioned command to destroy cluster by specifying installation directory.
❯ ./openshift-install destroy cluster --dir ocp413-rc6 --log-level=debugThis should destroy all resources created for cluster. If you have provisioned other resources in the generated subnet, the destroy command will fail.
Notes
- You can use pre-provisioned VPC see https://docs.openshift.com/container-platform/4.12/installing/installing_ibm_cloud_public/installing-ibm-cloud-vpc.html#installing-ibm-cloud-vpc
- Cloud credential request – An admin will have to create these for you, and as such, you’ll need to copy them over to the right locations in manifests/
- use
--log-level debugwith the installer to inspect the run.
References
-
Weekly Notes
There are so many interesting things to share:
- google/go-containerregistry has some super helpful tools, in fact I raised a PR to make sure they build ppc64le binaries #1680
crane is a tool for interacting with remote images and registries.
You can extract a binary
my-utilfor a given architecture using:crane export ppc64le/image-id:tag image.tar tar xvf image.tar bin/my-utilYou can extract a binary from a manifest-listed image using:
crane export --platform ppc64le image-id:tag image.tar tar xvf image.tar bin/my-util- I found ko which enables multiarch builds (a complete manifest list image).
- Quickly checking manifest-list image’s supported architectures
podman manifest inspect registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 | jq -r '.manifests[].platform.architecture' amd64 arm arm64 ppc64le s390x- My team tagged new releases for:
a. IBM/powervs-tang-server-automation: v1.0.4 b. IBM/powervm-tang-server-automation: v1.0.0