Paul Bastide

Blog

  • Multi Arch Compute OpenShift Container Platform (OCP) cluster on IBM Power 

    Following the release of Red Hat OpenShift 4.14, clients can run x86 and IBM Power Worker Nodes in the same OpenShift Container Platform Cluster with Multi-Architecture Compute. A study compared the performance implications of deploying applications on a Multi Arch Compute OpenShift Container Platform (OCP) cluster with a cluster exclusively built on IBM Power architecture. Findings revealed that performance had no significant impact with or without Multi Arch Compute. Click here to learn more about the study and the results found. 

    Watch the Red Hat OpenShift Multi-Arch Introduction Video to learn how, why, and when to add Power to your x86 OpenShift cluster.   

    Watch the OpenShift Multi-Arch Sock Shop Demonstration Video deploying the open-source Sock Shop e-commerce solution using a mix of x86 and Power Worker Nodes with Red Hat OpenShift Multi-Arch to further your understanding. 

  • Awesome Notes – 11/28

    Here are some great resources for OpenShift Container Platform on Power:

    UKI Brunch & Learn – Red Hat OpenShift – Multi-Architecture Compute

    Glad to see the Multiarchitecture Compute with an Intel Control Plane and Power worker in all its glory. Thanks to Paul Chapman

    https://www.linkedin.com/posts/chapmanp_uki-brunch-learn-red-hat-openshift-activity-7133370146890375168-AmuL?utm_source=share&utm_medium=member_desktop

    Explore Multi Arch Compute in OpenShift cluster with IBM Power systems

    In the ever-evolving landscape of computing, the quest for optimal performance and adaptability remains constant. This study delves into the performance implications of deploying applications on a Multi Arch Compute OpenShift Container Platform (OCP) cluster, comparing it with a cluster exclusively built on IBM Power architecture. Our findings reveal that, with or without Multi Arch Compute, there is no significant impact on performance.

    Thanks to @Mel from the IBM Power Systems Performance Team

    https://community.ibm.com/community/user/powerdeveloper/blogs/mel-bakhshi/2023/11/28/explore-mac-ocp-on-power

    Enabling FIPS Compliance in Openshift Cluster Platform on Power

    A new PDEX blog is posted to help the technical experts configure their OpenShift Container Platform on Power and the necessary background to configure FIPS 140-2 compliance.

    https://community.ibm.com/community/user/powerdeveloper/blogs/paul-bastide/2023/11/21/enabling-fips-compliance-in-openshift-cluster-plat?CommunityKey=daf9dca2-95e4-4b2c-8722-03cd2275ab63

    Encrypting etcd data on OpenShift Container Platform on Power

    This article was originally posted to Medium by Gaurav Bankar and has been updated.

    And now is posted with updated details for 4.14.

    https://community.ibm.com/community/user/powerdeveloper/blogs/paul-bastide/2023/11/21/encrypting-etcd-data-on-power?CommunityKey=daf9dca2-95e4-4b2c-8722-03cd2275ab63

    Using TLS Security Profiles on OpenShift Container Platform on IBM Power

    This article identifies using cluster operators and components with TLS Security profiles, covers the available security profiles, and how to configure each profile, and verify each profile is properly enabled.

    https://community.ibm.com/community/user/powerdeveloper/communities/community-home/recent-community-blogs?communitykey=daf9dca2-95e4-4b2c-8722-03cd2275ab63

    Encrypting disks on OpenShift Container Platform on Power Systems

    This document outlines the concepts, how to setup an external tang cluster on IBM PowerVS, how to setup a cluster on IBM PowerVS and how to confirm the encrypted disk setup.

    https://community.ibm.com/community/user/powerdeveloper/blogs/paul-bastide/2023/11/21/encrypting-disks-on-openshift-container-platform-o?CommunityKey=daf9dca2-95e4-4b2c-8722-03cd2275ab63

    Configuring a PCI-DSS compliant OpenShift Container Platform cluster on IBM Power

    This article outlines how to verify the profiles, check for the scan results, and configure a compliant cluster.

    https://community.ibm.com/community/user/powerdeveloper/blogs/paul-bastide/2023/11/21/configuring-a-pci-dss-compliant-openshift-containe?CommunityKey=daf9dca2-95e4-4b2c-8722-03cd2275ab63

    Open Source Container images for Power now available in IBM Container Registry

    The OpenSource team has posted new images:

    grafana-mimir-build-image2.9.0docker pull icr.io/ppc64le-oss/grafana-mimir-build-image-ppc64le:2.9.0Nov 24, 2023
    grafana-mimir-continuous-test2.9.0docker pull icr.io/ppc64le-oss/grafana-mimir-continuous-test-ppc64le:2.9.0Nov 24, 2023
    grafana-mimir2.9.0docker pull icr.io/ppc64le-oss/grafana-mimir-ppc64le:2.9.0Nov 24, 2023
    grafana-mimir-rules-action2.9.0docker pull icr.io/ppc64le-oss/grafana-mimir-rules-action-ppc64le:2.9.0Nov 24, 2023
    grafana-mimirtool2.9.0docker pull icr.io/ppc64le-oss/grafana-mimirtool-ppc64le:2.9.0Nov 24, 2023
    grafana-query-tee2.9.0docker pull icr.io/ppc64le-oss/grafana-query-tee-ppc64le:2.9.0Nov 24, 2023
    filebrowserv2.24.2docker pull icr.io/ppc64le-oss/filebrowser-ppc64le:v2.24.2Nov 24, 2023
    neo4j5.9.0docker pull icr.io/ppc64le-oss/neo4j-ppc64le:5.9.0Nov 24, 2023
    kong3.3.0docker pull icr.io/ppc64le-oss/kong-ppc64le:3.3.0Nov 24, 2023
    https://community.ibm.com/community/user/powerdeveloper/blogs/priya-seth/2023/04/05/open-source-containers-for-power-in-icr

    Multi-arch build pipelines for Power: Automating multi-arch image builds

    Multi-arch build pipelines can greatly reduce the complexity of supporting multiple operating systems and architectures. Notably, images built on the Power architecture can seamlessly be supported by other architectures, and vice versa, amplifying the versatility and impact of your applications. Furthermore, automating the processes using various CI tools, not only accelerates the creation of multi-arch images but also ensures consistency, reliability, and ease of integration into diverse software environments.

    Building on our exploration of multi-arch pipelines for IBM Power in the first blog, this blog delves into the next frontier: Automation. Automating multi-arch image builds using Continuous Integration (CI) tools has become essential in modern software development. This process allows developers to efficiently create and maintain container images that can run on various CPU architectures, such as IBM Power (ppc64le), x86 (amd64), or ARM ensuring compatibility across diverse hardware environments.

    Part 1 https://community.ibm.com/community/user/powerdeveloper/blogs/prajyot-parab/2023/11/27/multi-arch-pipelines-for-ibm-power Part 2 https://community.ibm.com/community/user/powerdeveloper/blogs/prajyot-parab/2023/11/27/automating-multi-arch-image-builds-for-power

  • Quay.io now available on IBM Power Systems

    Thanks to the RH and Power Team and Yussuf in particular – IBM Power now has quay.io install-run support.

    Red Hat Quay is a distributed, highly available, security-focused, and scalable private image registry platform that enables you to build, organize, distribute, and deploy containers for your enterprise. It provides a single and resilient content repository for delivering containerized software to development and production across Red Hat OpenShift and Kubernetes clusters.

    Now, Red Hat Quay is available on IBM Power with version 3.10. Read the official Red Hat Quay 3.10 blog and for more information visit the Red Hat Quay Documentation page.

    https://community.ibm.com/community/user/powerdeveloper/blogs/yussuf-shaikh/2023/11/07/quay-on-power

  • Notes

    Here are my notes from the week:

    Announcement of OpenShift 4.14 on Power

    Power Developer Exchange: Red Hat OpenShift 4.14 Now Available on IBM Power

    IBM® is very excited to announce that Red Hat OpenShift 4.14 has been released and is available to run natively on IBM Power.

    Multi-Architecture Compute

    With Red Hat OpenShift 4.14, Multi-Architecture Compute comes to IBM Power and IBM Z platforms. Multi-Architecture Compute provides a single heterogeneous cluster, enabling fit-for-purpose computing, so that customers can align tasks and applications to CPU strengths and software availability rather than to one architecture. This also helps reduce the cost and complexity of solutions that require multiple architectures. 

    https://community.ibm.com/community/user/powerdeveloper/blogs/brandon-pederson1/2023/11/08/red-hat-openshift-414-now-available-on-ibm-power

    Brunch and Learn with MAC

    Paul Chapman is hosting a lunch and learn on Multi-architecture Compute. It’s a good session to join.

    Hey, would you like to join me for this short brunch and learn? I will discuss and demonstrate Red Hat OpenShift Multi-Architecture Computing (MAC).

MAC now supports x86, ARM, and Power Worker Nodes, all within the same Red Hat OpenShift Cluster. It should be an exciting and informative event. I'd love to have your company and hear your feedback.

–10:30 GMT/UTC 17th November 2023
–Register here https://ibm.biz/BdPYQH

#powermod
#power10 #ibmpowersystems #ibmpowersystemsvirtualservers
#redhatopenshiftcontainerplatform #redhatopenshift
#containersolutions #containers
    https://ibm.biz/BdPYQH

    Butane with Key File

    On a linux intel or arm machine, y ou can run the following:

    1. Create the butane alias
    alias butane='podman run --rm --interactive         \
              --security-opt label=disable          \
              --volume "$(pwd)":/pwd --workdir /pwd \
              quay.io/coreos/butane:latest'
    1. Generate the key_file contents
    openssl genrsa -out out.rsa 4096
    1. Use this file
    cat <<EOF > 99-worker-luks.bu
variant: openshift
version: 4.14.0
metadata:
  name: 99-worker-luks
  labels:
    machineconfiguration.openshift.io/role: worker
storage:
    luks:
    - name: luks-encrypt-worker
      device: /dev/disk/by-partlabel/root
      key_file:
        source: data:;base64,$(cat out.rsa | gzip - | base64 --wrap=0)
        compression: gzip
EOF
    1. Run the Butane command
    butane 99-worker-luks.bu -o 99-worker-luks.yaml

    It generates:

    # Generated by Butane; do not edit
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-luks
spec:
  config:
    ignition:
      version: 3.4.0
    storage:
      luks:
        - device: /dev/disk/by-partlabel/root
          keyFile:
            compression: gzip
            source: data:;base64,<EXCLUDED>
          name: luks-encrypt-worker

    You probably need to consider:

    luks:
   enabled: false
   filesystem_device: /dev/mapper/root
   format: xfs
   wipeFileSystem: true
   device: /dev/disk/by-partlabel/root
   label: luks-root
   options:
      - --cipher
      - aes-cbc-essiv:sha256
   wipeVolume: true
   name: root

    The cipher options probably needs some work.

    Ref: https://github.com/ocp-power-automation/ocp4-playbooks/blob/efb74c59edfc8de2f2579f7b754463a718028c56/playbooks/roles/ocp-config/templates/99-node-machineconfig.yaml.j2#L14 

    Ref: https://github.com/ocp-power-automation/ocp4-playbooks/blob/efb74c59edfc8de2f2579f7b754463a718028c56/playbooks/roles/ocp-config/defaults/main/main.yaml#L23 

    Ref: https://coreos.github.io/butane/upgrading-openshift/ 

    Ref: https://coreos.github.io/butane/config-openshift-v4_14/ 

    Ref: https://coreos.github.io/butane/examples/

    List of Red Hat Products released in October 2023

    In the month of October, the following Red Hat products were released on IBM Power. 

    https://community.ibm.com/community/user/powerdeveloper/blogs/ashwini-sule/2023/11/09/red-hat-products-oct-2023

  • Notes from the Week

    A few updates this week are:

    Red Hat OpenShift Container Platform 4.14.0 for Power Systems

    The Red Hat team released OpenShift Container Platform with new support for Power Systems.

    The features I worked on with my team are:

    The installer and client are available at link

    Hack to List Interfaces

    # ip -j -4 -o address  | jq -r '.[].addr_info[] | select(.dev == "env2").local'
192.168.166.70

    systemd Commands Cheat Sheet

    Users and administrators query and control systemd behavior through the systemctl command. This systemd Cheat Sheet presents the most common uses of systemctl, along with journalctl for displaying information about systemd activities from its logs.

    https://developers.redhat.com/cheat-sheets/systemd-commands-cheat-sheet?sc_cid=7013a000003SyUMAA0

    The systemd cheat sheet is a good reference. Very comprehensive.

    Posts on OCP 4.14

    Paul Chapman updated the community on the new features in 4.14.

    https://www.linkedin.com/posts/chapmanp_openshift-container-platform-414-release-activity-7125400057436073984-68Ju?utm_source=share&utm_medium=member_desktop

  • Notes of the Week

    1. Updated Open Source Container images for Power now available in IBM Container Registry

    The IBM Linux on Power team posted an updated container list including new versions of MongoDB

    https://community.ibm.com/community/user/powerdeveloper/blogs/priya-seth/2023/04/05/open-source-containers-for-power-in-icr

    2. The Acme Air application that uses multiarchitecture compute is moved on quay and github.

    The source code is at https://github.com/ocp-power-demos/acmeair-multiarchitecture-compute

    The images are at https://quay.io/repository/powercloud/acme-airlines-mac?tab=tags

    These images are helpful when exploring multiarch compute.

    3. YouTube: What’s new in OpenShift 4.14

    This video outlines what’s new in OpenShift Container Platform 4.14.
    https://www.youtube.com/watch?v=T0Je1UMqRNc

    4. Paul Chapman’s Invitation to IBM Power’s Multi-Architecture Cluster Early Adoption & Feedback Program

    This site has a short introduction and call to action. It’s great to see videos on Multiarchitecture Compute.

    https://www.linkedin.com/posts/chapmanp_invitation-to-ibm-powers-multi-architecture-activity-7119940146019328002-5tjH

  • Useful Notes for September and October 2023

    Hi everyone, I’ve been heads down working on Multiarchitecture Compute and the Power platform for IBM.

    How to add /etc/hosts file entries in OpenShift containers

    You can add host aliases into the Pod Definition which is handy if the code is hard coded with a DNS entry. 

          hostAliases:
      - ip: "127.0.0.1"
        hostnames:
        - "home"
     - ip: "10.1.x.x"
        hostnames:
        - "remote-host"
    https://access.redhat.com/solutions/3696301

    Infrastructure Nodes in OpenShift 4

    A link to Infra nodes which provide a specific role in the cluster.

    https://access.redhat.com/solutions/5034771

    Multiarchitecture Compute Research

    Calling all IBM Power customers looking to impact Power modernization capabilities. The IBM Power Design Team is facilitating a study to understand customer sentiment toward Multi-Architecture Computing (MAC) and needs your help.

    https://community.ibm.com/community/user/powerdeveloper/blogs/erica-albert/2023/10/11/multi-architecture-computing-research-recruit 

    This is an interesting opportunity to work with customers on IBM Power and OpenShift as they mix the architecture workloads to meet their needs.

  • Weekly Notes

    Here are my weekly notes:

    Flow Connector

    If you are using the VPC, you can track connections between your subnets and your VPC using Flow Connector.

    ❯ find . -name “*.gz” -exec gunzip {} \;

    ❯ grep -Rh 192.168.200.10 | jq -r ‘.flow_logs[] | select(.action == “rejected”) | “\(.initiator_ip),\(.target_ip),\(.target_port)”‘ | sort -u | grep 192.168.200.10

    10.245.0.5,192.168.200.10,36416,2023-08-08T14:31:32Z

    10.245.0.5,192.168.200.10,36430,2023-08-08T14:31:32Z

    10.245.0.5,192.168.200.10,58894,2023-08-08T14:31:32Z

    10.245.1.5,192.168.200.10,10250,2023-08-08T14:31:41Z

    10.245.1.5,192.168.200.10,10250,2023-08-08T14:31:42Z

    10.245.1.5,192.168.200.10,9100,2023-08-08T14:31:32Z

    10.245.129.4,192.168.200.10,43524,2023-08-08T14:31:32Z

    10.245.64.4,192.168.200.10,10250,2023-08-08T14:31:32Z

    10.245.64.4,192.168.200.10,10250,2023-08-08T14:31:42Z

    10.245.64.4,192.168.200.10,9100,2023-08-08T14:31:42Z

    10.245.64.4,192.168.200.10,9537,2023-08-08T14:50:36Z

    Image Pruner Reports Error….

    You can check the image-registry status on the cluster operator.

    ❯ oc get co image-registry
image-registry                             4.14.0-ec.4   True        False         True       3d14h   ImagePrunerDegraded: Job has reached the specified backoff limit

    The cronjob probably failed, so we can check that it exists.

    ❯ oc get cronjob -n openshift-image-registry
NAME           SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
image-pruner   0 0 * * *   False     0        16h             3d15h

    We can run a one-off to clear the status above.

    ❯ oc create job --from=cronjob/image-pruner one-off-image-pruner -n openshift-image-registry
job.batch/one-off-image-pruner created

    Then your image-registry should be a-ok.

    Ref: https://gist.github.com/ryderdamen/73ff9f93cd61d5dd45a0c50032e3ae03

  • Weekly Notes

    Here are the very cool things I learned this week:

    CRI-O Graduated

    CRI-O has graduated at the CNCF – see the announcement Cloud Native Computing Foundation Announces Graduation of CRI-O. This points to the maturity of Cloud Native runtimes.

    Checking an Ignition on a Failed Instance

    I use PowerVS and had a bad ignition file, so I logged in via the Console in PowerVS. Then run journalctl -xe, and then mount the ignition file and cat it out.

  • Two ways to grab the Ignition for RHCOS/OCP4

    There are two ways to grab the ignition files for the workers in the cluster:

    1. A downloaded ignition file stored (in data folder) using curl:
    • curl -k http://api.demo.ocp-multiarch.xyz:22623/config/worker -o worker.ign -H "Accept: application/vnd.coreos.ignition+json;version=3.2.0"

    2. Download the ignition file using the oc commandline

    • oc extract -n openshift-machine-api secret/worker-user-data --keys=userData --to=-

    I’m adding this because I use it every day, and others might find it helpful.