Weekly Notes

Here are my notes from the week:

  1. Subnet to CIDR block Cheat Sheet
  2. OpenShift Installer Provisioned Infrastructure for IBM Cloud VPC

rfc1878: Subnet CIDR Cheat Sheet

I found a great cheat sheet for CIDR subnet masks.

   Mask value:                             # of
   Hex            CIDR   Decimal           addresses  Classfull    /1         2048 M     128 A
   C0.00.00.00    /2         1024 M      64 A
   E0.00.00.00    /3          512 M      32 A
   F0.00.00.00    /4          256 M      16 A
   F8.00.00.00    /5          128 M       8 A
   FC.00.00.00    /6           64 M       4 A
   FE.00.00.00    /7           32 M       2 A
   FF.00.00.00    /8           16 M       1 A
   FF.80.00.00    /9          8 M     128 B
   FF.C0.00.00   /10          4 M      64 B
   FF.E0.00.00   /11          2 M      32 B
   FF.F0.00.00   /12       1024 K      16 B
   FF.F8.00.00   /13        512 K       8 B
   FF.FC.00.00   /14        256 K       4 B
   FF.FE.00.00   /15        128 K       2 B
   FF.FF.00.00   /16         64 K       1 B
   FF.FF.80.00   /17       32 K     128 C
   FF.FF.C0.00   /18       16 K      64 C
   FF.FF.E0.00   /19        8 K      32 C
   FF.FF.F0.00   /20        4 K      16 C
   FF.FF.F8.00   /21        2 K       8 C
   FF.FF.FC.00   /22        1 K       4 C
   FF.FF.FE.00   /23      512         2 C
   FF.FF.FF.00   /24      256         1 C
   FF.FF.FF.80   /25    128       1/2 C
   FF.FF.FF.C0   /26     64       1/4 C
   FF.FF.FF.E0   /27     32       1/8 C
   FF.FF.FF.F0   /28     16      1/16 C
   FF.FF.FF.F8   /29      8      1/32 C
   FF.FF.FF.FC   /30      4      1/64 C
   FF.FF.FF.FE   /31      2     1/128 C
   FF.FF.FF.FF   /32      1

Thanks to the following sites for the clue to the rfc and the rfc.

Mutating WebHook to add Node Selectors

Thanks to these sites

  1. hmcts/k8s-env-injector provided inspiration for this approach and updates the code patterns for the latest kubernetes versions.
  2. phenixblue/imageswap-webhook provided the python based pattern for this approach.
  3. Kubernetes: MutatingAdmissionWebhook

I added some code to add annotations and nodeSelectors https://github.com/prb112/openshift-demo/tree/main/mutating

Installing OpenShift install provisioned infrastructure on IBM Cloud VPC

This document outlines installing the IPI IBMCloud using the openshift-installer.

As of OpenShift 4.13, you can install a cluster into an existing Virtual Private Cloud (VPC) on IBM Cloud VPC. The installation program provisions the required infrastructure, which you can then further customize.

This document describes the creation of OCP cluster using IPI (Installer Provisioned Infrastructure) on exiting IBM Cloud VPC.

This setup is used with the day-2 operations on PowerVS to make a multiarch compute cluster.

  1. Create IBM API Key
  2. Create the IAM Services
  3. Pick your build
  4. Deploy

1. Create IBM API Key

  1. Navigate to API keys iam – api keys
  2. Click Create
  3. Enter name rdr-demo
  4. Click Create
  5. Copy your API key, it’ll be used later on.

2. Create the IAM Services

  1. Navigate to Service Ids iam – serviceids
  2. click create service id with name rdr-demo to identify your team.
  3. assign access
Internet Services	All	Viewer, Operator, Editor, Reader, Writer, Manager, Administrator		--	
Cloud Object Storage	All	Viewer, Operator, Editor, Reader, Writer, Manager, Content Reader, Object Reader, Object Writer, Administrator		--	
IAM Identity Service	All	Viewer, Operator, Editor, Administrator, ccoctlPolicy, policycreate		--	
Resource group only	ocp-dev-resource-group resource group	Viewer, Administrator, Editor, Operator		--	
VPC Infrastructure Services	All	Viewer, Operator, Editor, Reader, Writer, Administrator, Manager

3. Pick your build

I used 4.13.0-rc.7.

4. Deploy

  1. Connect to your jumpserver or bastion where you are doing the deployment.

Tip: it’s worth having tmux installed for this install (it’ll take about 1h30m)

  1. Export the API KEY you created above
  1. Create a working folder
❯ mkdir -p ipi-vpc-414-rc7
❯ cd ipi-vpc-414-rc7
  1. Download the installers and extract to the binary folder.
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/ccoctl-linux.tar.gz
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-client-linux.tar.gz
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-install-linux.tar.gz
❯ tar xvf ccoctl-linux.tar.gz --dir /usr/local/bin/
❯ tar xvf openshift-client-linux.tar.gz --dir /usr/local/bin/
❯ tar xvf openshift-install-linux.tar.gz --dir /usr/local/bin/
  1. Verify the openshift-install version is correct.
❯ openshift-install version
openshift-install 4.13.0-rc.7
built from commit 3e0b2a2ec26d9ffcca34b361896418499ad9d603
release image quay.io/openshift-release-dev/ocp-release@sha256:aae5131ec824c301c11d0bf11d81b3996a222be8b49ce4716e9d464229a2f92b
release architecture amd64
  1. Copy over your pull-secret.

a. Login with your Red Hat id

b. Navigate to https://console.redhat.com/openshift/install/ibm-cloud 

c. Scroll down the page and copy the pull-secret.

This pull-secret should work for you and save for later as pull-secret.txt in the working directory.

  1. Extract the CloudControlsRequest objects and create the credentials.
RELEASE_IMAGE=$(openshift-install version | awk '/release image/ {print $3}')
oc adm release extract --cloud=ibmcloud --credentials-requests $RELEASE_IMAGE --to=rdr-demo
ccoctl ibmcloud create-service-id --credentials-requests-dir rdr-demo --output-dir rdr-demo-out --name rdr-demo --resource-group-name ocp-dev-resource-group
  1. Create the install-config
❯ openshift-install create install-config --dir rc7_2
? SSH Public Key /root/.ssh/id_rsa.pub                                                                     
? Platform ibmcloud                                                                                        
? Region jp-osa                                                                                            
? Base Domain ocp-multiarch.xyz (rdr-multi-is)                                                             
? Cluster Name rdr-multi-pb                                                                                
? Pull Secret [? for help] ********************************************************************************
INFO Manifests created in: rc7_1/manifests and rc7_1/openshift
  1. Edit the install-config.yaml to add resourceGroupName
    region: jp-osa
    resourceGroupName: my-resource-group 
  1. Copy the generated ccoctl manifests over.
❯ cp rdr-demo-out/manifests/* rc7_1/manifests/
  1. Create the manifests.
❯ openshift-install create manifests --dir=rc7_1
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Manifests created in: rc7_1/manifests and rc7_1/openshift
  1. Create the cluster.
❯ openshift-install create cluster --dir=rc7_3
INFO Consuming Worker Machines from target directory
INFO Consuming Common Manifests from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Master Machines from target directoryINFO Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.13-9.2/builds/413.92.202305021736-0/x86_64/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2.gz?sha256=222abce547c1bbf32723676f4977a3721c8a3788f0b7b6b3496b79999e8c60b3'                                   
INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2. Reusing...           INFO Creating infrastructure resources...
INFO Waiting up to 20m0s (until 12:09PM) for the Kubernetes API at https://api.xyz.ocp-multiarch.xyz:6443... 
INFO API v1.26.3+b404935 up                       
INFO Waiting up to 30m0s (until 12:19PM) for bootstrapping to complete... 
INFO Destroying the bootstrap resources...        
INFO Waiting up to 40m0s (until 12:41PM) for the cluster at https://api.xyz.ocp-multiarch.xyz:6443 to initialize... 
INFO Checking to see if there is a route at openshift-console/console... 
INFO Install complete!                            
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/ipi-vpc-414-rc7/rc7_3/auth/kubeconfig' 
INFO Access the OpenShift web-console here: 
INFO Login to the console with user: "kubeadmin", and password: "xxxxxxxxx-wwwwww-xxxx-aas" 
INFO Time elapsed: 1h28m9s      
  1. Verify the cluster

a. set kubeconfig provided by installation

export KUBECONFIG=$(pwd)/rc7_1/auth/kubeconfig

b. Check the nodes are Ready

❯  oc get nodes
NAME                                    STATUS   ROLES          AGE     		VERSION
rdr-multi-ca-rc6-tplwd-master-0             Ready    control-plane,master  5h13m   v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-master-1             Ready    control-plane,master  5h13m   v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-master-2             Ready    control-plane,master  5h13m   v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-pfqjx  Ready    worker                 	4h47m   v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-th8j4  Ready    worker                 4h47m   v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-xl75m Ready    worker                 4h53m   v1.26.3+b404935

c. Check Cluster Operators

❯ oc get co
NAME                                       	VERSION       AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
authentication                             	4.13.0-rc.6   True        False         False      4h43m
baremetal                              	4.13.0-rc.6   True        False         False      5h5m
cloud-controller-manager            4.13.0-rc.6   True        False         False      5h13m
cloud-credential                           	4.13.0-rc.6   True        False         False      5h18m
cluster-autoscaler                         	4.13.0-rc.6   True        False         False      5h5m
config-operator                      	4.13.0-rc.6   True        False         False      5h7m
console                           	       	4.13.0-rc.6   True        False         False      4h47m
control-plane-machine-set     	4.13.0-rc.6   True        False         False      5h5m
csi-snapshot-controller                 4.13.0-rc.6   True        False         False      4h54m
dns                                        	4.13.0-rc.6   True        False         False      4h54m
etcd                                       	4.13.0-rc.6   True        False         False      4h57m
image-registry                             	4.13.0-rc.6   True        False         False      4h50m
ingress                                    	4.13.0-rc.6   True        False         False      4h51m
insights                                   	4.13.0-rc.6   True        False         False      5h
kube-apiserver                             	4.13.0-rc.6   True        False         False      4h53m
kube-controller-manager             4.13.0-rc.6   True        False         False      4h53m
kube-scheduler                             	4.13.0-rc.6   True        False         False      4h52m
kube-storage-version-migrator   4.13.0-rc.6   True        False         False      4h54m
machine-api                                	4.13.0-rc.6   True        False         False      4h48m
machine-approver                         4.13.0-rc.6   True        False         False      5h5m
machine-config                             	4.13.0-rc.6   True        False         False      5h6m
marketplace                                	4.13.0-rc.6   True        False         False      5h5m
monitoring                                 	4.13.0-rc.6   True        False         False      4h45m
network                                    	4.13.0-rc.6   True        False         False      5h8m
node-tuning                                	4.13.0-rc.6   True        False         False      4h54m
openshift-apiserver                       4.13.0-rc.6   True        False         False      4h47m
openshift-controller-manager     4.13.0-rc.6   True        False         False      4h54m
openshift-samples                         4.13.0-rc.6   True        False         False      4h50m
operator-lifecycle-manager         4.13.0-rc.6   True        False         False      5h6m
operator-lifecycle-manager-catalog         4.13.0-rc.6   True        False         False      5h6m
operator-lifecycle-manager-packageserver   4.13.0-rc.6   True        False         False      4h51m
service-ca                                 	4.13.0-rc.6   True        False         False      5h7m
storage                                    	4.13.0-rc.6   True        False         False      4h51m

Note – Confirm that all master/worker nodes and operators are running healthy and true.

  1. Verify the browser login

A. Open Browser and Login to Console URL using available credentials. e.g.,

URL - https://console-openshift-console.apps.xxxxxx.ocp-multiarch.xyz
	Username – kubeadmin
	Password - <Generated Password>
  1. destroy cluster Fire below mentioned command to destroy cluster by specifying installation directory.
❯ ./openshift-install destroy cluster --dir  ocp413-rc6 --log-level=debug

This should destroy all resources created for cluster. If you have provisioned other resources in the generated subnet, the destroy command will fail.


  1. You can use pre-provisioned VPC see https://docs.openshift.com/container-platform/4.12/installing/installing_ibm_cloud_public/installing-ibm-cloud-vpc.html#installing-ibm-cloud-vpc
  2. Cloud credential request – An admin will have to create these for you, and as such, you’ll need to copy them over to the right locations in manifests/
  3. use --log-level debug with the installer to inspect the run.


  1. installing on ibm cloud vpc
  2. create service id
  3. Exporting the IBM Cloud VPC API key




Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.