Blog

To run multiple OpenShift clusters from one bastion requires managing dhcpd, named, http, haproxy with isolated configurations.

After deploying with ocp4-upi-powervm, you can ‘move’ the configuration over

1. dhcpd enables booting the rhcos nodes, which then can grab their configuration. dhcpd support include statements, allowing you to split subnets, host reservations, and cluster-specific configurations into separate files.

   1. Create the conf.d directory: mkdir -p /etc/dhcp/conf.d
   2. Modify your main /etc/dhcp/dhcpd.conf to include the directory. Add this at the bottom of the file: include "/etc/dhcp/conf.d/ocp-cluster-1.conf";
   3. Create the file /etc/dhcp/conf.d/ocp-cluster-1.conf – you’ll have to give the host unique names.

   subnet 10.20.176.0 netmask 255.255.240.0 {
   interface eth0;
       # Static entries
       host bootstrap { hardware ethernet fa:16:3e:ff:b7:b2; fixed-address 10.20.188.84; }
       host master-0 { hardware ethernet fa:16:3e:9b:c5:89; fixed-address 10.20.188.206; }
       host master-1 { hardware ethernet fa:16:3e:b7:ba:16; fixed-address 10.20.188.62; }
       host master-2 { hardware ethernet fa:16:3e:14:2c:ff; fixed-address 10.20.188.166; }
       host worker-0 { hardware ethernet fa:16:3e:97:7b:1b; fixed-address 10.20.188.79; }
       host worker-1 { hardware ethernet fa:16:3e:62:39:fe; fixed-address 10.20.188.234; }
       host worker-2 { hardware ethernet fa:16:3e:23:54:0a; fixed-address 10.20.188.131; }
       # this will not give out addresses to hosts not listed above
       #deny unknown-clients;
   
       # this is PXE specific
       filename "boot/grub2/powerpc-ieee1275/core.elf";
   
       next-server 10.20.188.128;
       }
   

   4. Restart the systemd service systemctl restart dhcpd

2. If you are hosting ignition files on httpd on port 8080.

   1. Create the ignition folder mkdir -p /var/www/html/ignition/{ocp-cluster-1,ocp-cluster-2}
   2. Copy the ignition files into /var/www/html/ignition/ocp-cluster-#
   3. Or Download the ignitions curl -k -H "Accept: application/vnd.coreos.ignition+json;version=3.4.0" -o /var/www/html/ignition/power.ign https://api-int.XYZ.powervs-openshift-ipi.cis.ibm.net:22623/config/power
   4. Restore selinux restorecon -r /var/www/html/ignition

3. HAProxy allows us to use separate use_backend and acl

   1. Edit /etc/haproxy/haproxy.cfg
   2. Add acl for the domain name based on hostname

   frontend https-all
   mode        tcp
   option      tcplog
   
   bind        *:443
   
   acl 02-https-ci req_ssl_sni -m end .mycluster1.ibm.net
   use_backend https-workers-02 if 02-https-ci
   
   acl 03-https req_ssl_sni -m end .mycluster2.ibm.net
   use_backend https-workers-03 if 03-https
   

   3. Create a backend target for the above:

   backend https-workers-03
   mode        tcp
   balance     roundrobin
   server      master1 192.168.3.11:443 check
   server      master2 192.168.3.12:443 check
   server      master3 192.168.3.13:443 check
   server      worker1 192.168.3.51:443 check
   server      worker2 192.168.3.52:443 check
   

We use this approach in OCP LibVirt CI see haproxy_C155F2U31.cfg

4. named support multiple conf files using the include directive

   1. Create the modular directory: mkdir -p /etc/named/conf.d
   2. Modify /etc/named.conf to include your custom zone files. include "/etc/named/conf.d/ocp-cluster-1.conf";
   3. Create the file /etc/named/conf.d/ocp-cluster-1.conf

   zone "mycluster2.ibm.net" IN {
       type master;
       file "/var/named/zones/db.ocp-cluster-1.local";
       allow-query { any; };
   };
   
   zone "122.168.192.in-addr.arpa" IN {
       type master;
       file "/var/named/zones/ocp-cluster-1.192.168.122";
       allow-query { any; };
   };
   

Using this approach you’ll be able to share the bastion.