Paul Bastide

OpenShift on Power

A list of useful Tips and useful links are on this page.

Tips

Tip 1: If you fail on setup, check the networking is open on the private network. You can check the Enable Communication Over the Private Network topic Option 2.

Describe a Resource
● oc explain pod –recursive
● oc explain pod.spec.containers

Links

  1. Power Systems Virtual Server

    • Create Subnets
    • Create Logical Partitions on ppc64le architecture
    • Load Boot Images

  2. ppc64le-cloud/pvsadm is a tool for managing IBM Power Systems Virtual Servers.

  3. OpenShift User Provided Infrastructure Install provides Terraform based automation code to help with the deployment of OpenShift Container Platform (OCP) 4.x on IBM Power Systems Virtual Server on IBM Cloud.

export VERSION=4.10
git clone --single-branch --branch release-$VERSION \
     https://github.com/ocp-power-automation/ocp4-upi-powervs.git

  1. ocp-power-automation/ocp4-upi-powervs A set of scripts, from my colleague Rafael, to manage PowerVS.

  2. OpenShift Container Platform 4.x Tested Integrations (for ppc64le) lists the tested integrations.

  3. Deploying Red Hat OpenShift Container Platform 4.x on IBM Power Systems Virtual Servers detailed learning path for installing on PowerVS

Cross-compiling and Running Multi architecture builds

Here are some quick tips:

Rebooting a Worker Node

While the preferred method is to sequentially go through the Reboot Steps. The following is the dirty method to reboot a node and simulate a disruption.

  1. Login to OpenShift
# oc login --token=sha256~xyz --server=https://api.xyz.sslip.io:6443
Logged into "https://api.xyz.sslip.io:6443" as "kube:admin" using the token provided.
Using project "default".
  1. List the worker nodes
$ oc get nodes -lnode-role.kubernetes.io/worker
NAME                                               STATUS   ROLES    AGE   VERSION
worker-0.xyz.sslip.io   Ready    worker   50d   v1.23.5+b0357ed
worker-1.xyz.sslip.io   Ready    worker   50d   v1.23.5+b0357ed
  1. Connect to the node
$ oc debug node/worker-0.xyz.sslip.io
Starting pod/worker-xyz-debug ...
To use host binaries, run `chroot /host`
Pod IP: 192.168.0.90
If you don't see a command prompt, try pressing enter.
  1. Change the root
$ chroot /host
  1. Reboot the node
$ systemctl reboot

Wait for the node to come back up.

Terraform on Power

If you are using the Terraform install on Power, you’ll need to download it from unicamp.

$ curl -o /usr/bin/terraform -L https://github.com/ppc64le-development/terraform-ppc64le/releases/download/v1.3.4/terraform-v1.3.4
$ chmod +x /usr/bin/terraform

Then download the providers from IBM’s github repo: 

$ mkdir -p ~/terraform
$ cd terraform
$ curl -L https://github.com/ocp-power-automation/terraform-providers-power/releases/download/v0.11/archive.zip -o archive.zip
$ unzip archive.zip

Update the Terraform Plugins

$ terraform init --plugin-dir ~/terraform/

We then use ocp4-upi-powervm

$ set +o history
$ export POWERVC_USERNAME=xxxxxxxxxxxxxxx
$ export POWERVC_PASSWORD=xxxxxxxxxxxxxxx
$ export RHEL_SUBS_USERNAME=xxxxxxxxxxxxxxx
$ export RHEL_SUBS_PASSWORD=xxxxxxxxxxxxxxx
$ set -o history
$ terraform apply -var-file var.tfvars -var user_name="$POWERVC_USERNAME" -var password="$POWERVC_PASSWORD" -var rhel_subscription_username="$RHEL_SUBS_USERNAME" -var rhel_subscription_password="$RHEL_SUBS_PASSWORD"

Assign Pod to Node using oc run

I had to run a command to create a 1000 pods on a master node. Here is the little hack for it: 

for NS in $(seq 1 1000)
do
    oc run pause-$NS --image=docker.io/ibmcom/pause:3.1 --overrides='{"apiVersion": "v1", "spec": {"nodeSelector": { "kubernetes.io/hostname": "master-0.ocp-power.xyz" }}}'
done

OpenShift – Add a user as a Cluster Admin

  1. Navigate to https://cloud.ibm.com/

    1. Login to ibmcloud using the commandline tool
$ ibmcloud login --sso

  1. Select account your development account

  2. Configure your cluster with your cluster id

$ ibmcloud oc cluster config -c ex-ocp-lon06 --admin
  1. Type oc login (it’ll tell you where to request the oauth token)
$ oc login
You must obtain an API token by visiting https://c108-e.eu-gb.containers.cloud.ibm.com:31344/oauth/token/request

  1. Navigate to https://c108-e.eu-gb.containers.cloud.ibm.com:31344/oauth/token/request It may prompt you to login and then Display Token

  2. Login to OpenShift

$ oc login --token=sha256~aaa --server=https://c108-e.eu-gb.containers.cloud.ibm.com:31609
  1. List the users in your cluster
$ oc get users
NAME                             UID                                    FULL NAME   IDENTITIES
IAM#d3@ibm.com    3-a5a8-4ca4-2-1               IAM:IBMid-111
IAM#d2@ibm.com   3-61e9-4cc0-3-34               IAM:IBMid-112
IAM#d1@us.ibm.com          3-4c66-44a4-5-2               IAM:IBMid-113
  1. Add the user to the Cluster Role cluster-admin
$ oc adm policy add-cluster-role-to-user cluster-admin 'IAM#d1@us.ibm.com'
clusterrole.rbac.authorization.k8s.io/cluster-admin added: "IAM#d1@us.ibm.com"

Reference

PowerVS Regions

A great location to see which regions are supported with PowerVS and OpenShift Installer Provisioned Infrastructure, you can look at powervs_regions.go.

Security Profiles Operator

If you are working with highly secure container environments, you should consider the Security Profiles Operator. There is a great blog from Jacob that describes it in more detail and why you should care and use seccomp. link