Here are my notes from the week:
- Subnet to CIDR block Cheat Sheet
- OpenShift Installer Provisioned Infrastructure for IBM Cloud VPC
rfc1878: Subnet CIDR Cheat Sheet
I found a great cheat sheet for CIDR subnet masks.
Mask value: # of
Hex CIDR Decimal addresses Classfull
80.00.00.00 /1 128.0.0.0 2048 M 128 A
C0.00.00.00 /2 192.0.0.0 1024 M 64 A
E0.00.00.00 /3 224.0.0.0 512 M 32 A
F0.00.00.00 /4 240.0.0.0 256 M 16 A
F8.00.00.00 /5 248.0.0.0 128 M 8 A
FC.00.00.00 /6 252.0.0.0 64 M 4 A
FE.00.00.00 /7 254.0.0.0 32 M 2 A
FF.00.00.00 /8 255.0.0.0 16 M 1 A
FF.80.00.00 /9 255.128.0.0 8 M 128 B
FF.C0.00.00 /10 255.192.0.0 4 M 64 B
FF.E0.00.00 /11 255.224.0.0 2 M 32 B
FF.F0.00.00 /12 255.240.0.0 1024 K 16 B
FF.F8.00.00 /13 255.248.0.0 512 K 8 B
FF.FC.00.00 /14 255.252.0.0 256 K 4 B
FF.FE.00.00 /15 255.254.0.0 128 K 2 B
FF.FF.00.00 /16 255.255.0.0 64 K 1 B
FF.FF.80.00 /17 255.255.128.0 32 K 128 C
FF.FF.C0.00 /18 255.255.192.0 16 K 64 C
FF.FF.E0.00 /19 255.255.224.0 8 K 32 C
FF.FF.F0.00 /20 255.255.240.0 4 K 16 C
FF.FF.F8.00 /21 255.255.248.0 2 K 8 C
FF.FF.FC.00 /22 255.255.252.0 1 K 4 C
FF.FF.FE.00 /23 255.255.254.0 512 2 C
FF.FF.FF.00 /24 255.255.255.0 256 1 C
FF.FF.FF.80 /25 255.255.255.128 128 1/2 C
FF.FF.FF.C0 /26 255.255.255.192 64 1/4 C
FF.FF.FF.E0 /27 255.255.255.224 32 1/8 C
FF.FF.FF.F0 /28 255.255.255.240 16 1/16 C
FF.FF.FF.F8 /29 255.255.255.248 8 1/32 C
FF.FF.FF.FC /30 255.255.255.252 4 1/64 C
FF.FF.FF.FE /31 255.255.255.254 2 1/128 C
FF.FF.FF.FF /32 255.255.255.255 1
Thanks to the following sites for the clue to the rfc and the rfc.
Mutating WebHook to add Node Selectors
Thanks to these sites
- hmcts/k8s-env-injector provided inspiration for this approach and updates the code patterns for the latest kubernetes versions.
- phenixblue/imageswap-webhook provided the python based pattern for this approach.
- Kubernetes: MutatingAdmissionWebhook
I added some code to add annotations and nodeSelectors https://github.com/prb112/openshift-demo/tree/main/mutating
Installing OpenShift install provisioned infrastructure on IBM Cloud VPC
This document outlines installing the IPI IBMCloud using the openshift-installer.
As of OpenShift 4.13, you can install a cluster into an existing Virtual Private Cloud (VPC) on IBM Cloud VPC. The installation program provisions the required infrastructure, which you can then further customize.
This document describes the creation of OCP cluster using IPI (Installer Provisioned Infrastructure) on exiting IBM Cloud VPC.
This setup is used with the day-2 operations on PowerVS to make a multiarch compute cluster.
- Create IBM API Key
- Create the IAM Services
- Pick your build
- Deploy
1. Create IBM API Key
- Navigate to
API keys iam – api keys
- Click
Create
- Enter name
rdr-demo
- Click
Create
- Copy your API key, it’ll be used later on.
2. Create the IAM Services
- Navigate to
Service Ids iam – serviceids
- click create service id with name
rdr-demo to identify your team.
- assign access
Internet Services All Viewer, Operator, Editor, Reader, Writer, Manager, Administrator --
Cloud Object Storage All Viewer, Operator, Editor, Reader, Writer, Manager, Content Reader, Object Reader, Object Writer, Administrator --
IAM Identity Service All Viewer, Operator, Editor, Administrator, ccoctlPolicy, policycreate --
Resource group only ocp-dev-resource-group resource group Viewer, Administrator, Editor, Operator --
VPC Infrastructure Services All Viewer, Operator, Editor, Reader, Writer, Administrator, Manager
3. Pick your build
I used 4.13.0-rc.7.
4. Deploy
- Connect to your jumpserver or bastion where you are doing the deployment.
Tip: it’s worth having tmux installed for this install (it’ll take about 1h30m)
- Export the API KEY you created above
❯ export IC_API_KEY=<REDACTED>
- Create a working folder
❯ mkdir -p ipi-vpc-414-rc7
❯ cd ipi-vpc-414-rc7
- Download the installers and extract to the binary folder.
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/ccoctl-linux.tar.gz
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-client-linux.tar.gz
❯ curl -O -L https://mirror.openshift.com/pub/openshift-v4/amd64/clients/ocp/4.13.0-rc.7/openshift-install-linux.tar.gz
❯ tar xvf ccoctl-linux.tar.gz --dir /usr/local/bin/
❯ tar xvf openshift-client-linux.tar.gz --dir /usr/local/bin/
❯ tar xvf openshift-install-linux.tar.gz --dir /usr/local/bin/
- Verify the openshift-install version is correct.
❯ openshift-install version
openshift-install 4.13.0-rc.7
built from commit 3e0b2a2ec26d9ffcca34b361896418499ad9d603
release image quay.io/openshift-release-dev/ocp-release@sha256:aae5131ec824c301c11d0bf11d81b3996a222be8b49ce4716e9d464229a2f92b
release architecture amd64
- Copy over your pull-secret.
a. Login with your Red Hat id
b. Navigate to https://console.redhat.com/openshift/install/ibm-cloud
c. Scroll down the page and copy the pull-secret.
This pull-secret should work for you and save for later as pull-secret.txt in the working directory.
- Extract the CloudControlsRequest objects and create the credentials.
RELEASE_IMAGE=$(openshift-install version | awk '/release image/ {print $3}')
oc adm release extract --cloud=ibmcloud --credentials-requests $RELEASE_IMAGE --to=rdr-demo
ccoctl ibmcloud create-service-id --credentials-requests-dir rdr-demo --output-dir rdr-demo-out --name rdr-demo --resource-group-name ocp-dev-resource-group
- Create the install-config
❯ openshift-install create install-config --dir rc7_2
? SSH Public Key /root/.ssh/id_rsa.pub
? Platform ibmcloud
? Region jp-osa
? Base Domain ocp-multiarch.xyz (rdr-multi-is)
? Cluster Name rdr-multi-pb
? Pull Secret [? for help] ********************************************************************************
***********************************
INFO Manifests created in: rc7_1/manifests and rc7_1/openshift
- Edit the install-config.yaml to add
resourceGroupName
platform:
ibmcloud:
region: jp-osa
resourceGroupName: my-resource-group
- Copy the generated ccoctl manifests over.
❯ cp rdr-demo-out/manifests/* rc7_1/manifests/
- Create the manifests.
❯ openshift-install create manifests --dir=rc7_1
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Manifests created in: rc7_1/manifests and rc7_1/openshift
- Create the cluster.
❯ openshift-install create cluster --dir=rc7_3
INFO Consuming Worker Machines from target directory
INFO Consuming Common Manifests from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Master Machines from target directoryINFO Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.13-9.2/builds/413.92.202305021736-0/x86_64/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2.gz?sha256=222abce547c1bbf32723676f4977a3721c8a3788f0b7b6b3496b79999e8c60b3'
INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/rhcos-413.92.202305021736-0-ibmcloud.x86_64.qcow2. Reusing... INFO Creating infrastructure resources...
INFO Waiting up to 20m0s (until 12:09PM) for the Kubernetes API at https://api.xyz.ocp-multiarch.xyz:6443...
INFO API v1.26.3+b404935 up
INFO Waiting up to 30m0s (until 12:19PM) for bootstrapping to complete...
INFO Destroying the bootstrap resources...
INFO Waiting up to 40m0s (until 12:41PM) for the cluster at https://api.xyz.ocp-multiarch.xyz:6443 to initialize...
INFO Checking to see if there is a route at openshift-console/console...
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/ipi-vpc-414-rc7/rc7_3/auth/kubeconfig'
INFO Access the OpenShift web-console here:
INFO Login to the console with user: "kubeadmin", and password: "xxxxxxxxx-wwwwww-xxxx-aas"
INFO Time elapsed: 1h28m9s
- Verify the cluster
a. set kubeconfig provided by installation
export KUBECONFIG=$(pwd)/rc7_1/auth/kubeconfig
b. Check the nodes are Ready
❯ oc get nodes
NAME STATUS ROLES AGE VERSION
rdr-multi-ca-rc6-tplwd-master-0 Ready control-plane,master 5h13m v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-master-1 Ready control-plane,master 5h13m v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-master-2 Ready control-plane,master 5h13m v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-pfqjx Ready worker 4h47m v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-th8j4 Ready worker 4h47m v1.26.3+b404935
rdr-multi-ca-rc6-tplwd-worker-1-xl75m Ready worker 4h53m v1.26.3+b404935
c. Check Cluster Operators
❯ oc get co
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.13.0-rc.6 True False False 4h43m
baremetal 4.13.0-rc.6 True False False 5h5m
cloud-controller-manager 4.13.0-rc.6 True False False 5h13m
cloud-credential 4.13.0-rc.6 True False False 5h18m
cluster-autoscaler 4.13.0-rc.6 True False False 5h5m
config-operator 4.13.0-rc.6 True False False 5h7m
console 4.13.0-rc.6 True False False 4h47m
control-plane-machine-set 4.13.0-rc.6 True False False 5h5m
csi-snapshot-controller 4.13.0-rc.6 True False False 4h54m
dns 4.13.0-rc.6 True False False 4h54m
etcd 4.13.0-rc.6 True False False 4h57m
image-registry 4.13.0-rc.6 True False False 4h50m
ingress 4.13.0-rc.6 True False False 4h51m
insights 4.13.0-rc.6 True False False 5h
kube-apiserver 4.13.0-rc.6 True False False 4h53m
kube-controller-manager 4.13.0-rc.6 True False False 4h53m
kube-scheduler 4.13.0-rc.6 True False False 4h52m
kube-storage-version-migrator 4.13.0-rc.6 True False False 4h54m
machine-api 4.13.0-rc.6 True False False 4h48m
machine-approver 4.13.0-rc.6 True False False 5h5m
machine-config 4.13.0-rc.6 True False False 5h6m
marketplace 4.13.0-rc.6 True False False 5h5m
monitoring 4.13.0-rc.6 True False False 4h45m
network 4.13.0-rc.6 True False False 5h8m
node-tuning 4.13.0-rc.6 True False False 4h54m
openshift-apiserver 4.13.0-rc.6 True False False 4h47m
openshift-controller-manager 4.13.0-rc.6 True False False 4h54m
openshift-samples 4.13.0-rc.6 True False False 4h50m
operator-lifecycle-manager 4.13.0-rc.6 True False False 5h6m
operator-lifecycle-manager-catalog 4.13.0-rc.6 True False False 5h6m
operator-lifecycle-manager-packageserver 4.13.0-rc.6 True False False 4h51m
service-ca 4.13.0-rc.6 True False False 5h7m
storage 4.13.0-rc.6 True False False 4h51m
Note – Confirm that all master/worker nodes and operators are running healthy and true.
- Verify the browser login
A. Open Browser and Login to Console URL using available credentials. e.g.,
URL - https://console-openshift-console.apps.xxxxxx.ocp-multiarch.xyz
Username – kubeadmin
Password - <Generated Password>
- destroy cluster Fire below mentioned command to destroy cluster by specifying installation directory.
❯ ./openshift-install destroy cluster --dir ocp413-rc6 --log-level=debug
This should destroy all resources created for cluster. If you have provisioned other resources in the generated subnet, the destroy command will fail.
Notes
- You can use pre-provisioned VPC see https://docs.openshift.com/container-platform/4.12/installing/installing_ibm_cloud_public/installing-ibm-cloud-vpc.html#installing-ibm-cloud-vpc
- Cloud credential request – An admin will have to create these for you, and as such, you’ll need to copy them over to the right locations in manifests/
- use
--log-level debug with the installer to inspect the run.
References
- installing on ibm cloud vpc
- create service id
- Exporting the IBM Cloud VPC API key
