Author: Paul

  • Release 4.2.2 – Notes

    My Team just released IBM FHIR Server 4.2.2. Other than the amazing things documented and released with the release tab, I learned a few things.

    Replace Tags

    If you need to replace tags, force it with fetch

    ~$ git fetch --tags -f
    From github.com:IBM/FHIR
    t [tag update] 4.1.0 -> 4.1.0
    t [tag update] 4.2.2 -> 4.2.2
    

    Rebuild the Validation Package

    export BUILD_TYPE=RELEASE
    export BUILD_VERSION=4.2.2
    bash build/release/version.sh
    
    mvn ${THREAD_COUNT} -ntp -B clean source:jar source:test-jar javadoc:jar \
    install -f fhir-parent -Pfhir-validation-distribution,fhir-ig-carin-\
    bb,fhir-ig-davinci-pdex-plan-net,fhir-ig-mcode,fhir-ig-us-core,deploy-\
    bintray -DskipTests -pl ../fhir-ig-davinci-pdex-plan-net/,../fhir-\
    validation -amd
    

    -amd keeps the build focused only on the necessary packages (not the full fhir-parent)

    Idempotent Execution of the Role Creation

    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRSERVER') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRSERVER'; END IF; END;\""

    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRBATCH') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRBATCH'; END IF; END;\""

    Shell Pipestatus

    Checking the Status of any command in a pipe, it was helpful in some automation where I had to wait on a jar to finish and check the output. Source

    Command

    curl -L https://google.com | grep response | tee response.txt
    RC=${PIPESTATUS[1]}
    echo $RC
    

    Output

    4
    

    Reference

  • Apache Nifi and IBM FHIR Server: InvokeHTTP and SSL

    A user who is integrating Apache Nifi and IBM FHIR Server asked how they get the SSL to work between the two, and here is a small recipe for you:

    1. List Keys
    keytool -list -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password -rfc
    

    Check to see if you have a default, if you do, go to step 2, else step 3.

    1. Change default
    keytool -changealias -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
       -storepass change-password -alias default -destalias old_default
    

    You can always double check with step 3.

    1. Create a new default with a distinguished name for your hostname (mine is host.docker.internal)
    keytool -genkey -keyalg RSA -alias default -keystore \
    fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password -validity 2000 -keysize 2048 -dname cn=host.docker.internal
    
    1. Confirm the lists of keys
    keytool -list -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password
    
    Keystore type: PKCS12
    Keystore provider: SUN
    
    Your keystore contains 2 entries
    
    old_default, May 15, 2020, PrivateKeyEntry,
    Certificate fingerprint (SHA-256): 9D:94:C2:F8:C1:51:9B:0F:21:50:4F:BB:60:A4:8A:3F:AF:C0:F0:13:C4:80:BE:A3:94:42:04:46:56:DB:D9:7B
    default, May 15, 2020, PrivateKeyEntry,
    Certificate fingerprint (SHA-256): 5B:38:D5:FD:7F:8A:80:60:12:CF:7F:61:C6:D6:C5:54:F3:FD:F8:80:34:58:A5:3F:1C:8F:2C:0A:42:85:C0:49
    

    Notice, the new key.

    1. Restart your app server to pick up the latest. Once restarted, proceed to next step.

    2. Confirm you see the subject is the one you need.

    curl -k https://localhost:9443 -v 2>&1 | grep -i subject
    *  subject: CN=host.docker.internal
    
    1. Start a nifi image
    docker run -p 8080:8080 --rm apache/nifi:latest bash
    
    1. Find the docker container id
    $ docker ps
    CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                                                   NAMES
    09d2a7395fa2        apache/nifi:latest   "../scripts/start.sh…"   7 seconds ago       Up 6 seconds        8000/tcp, 8443/tcp, 10000/tcp, 0.0.0.0:8080->8080/tcp   gracious_rosalind
    
    1. Copy the fhirKeystore.p12 (in this case we just updated this one only).
    docker cp fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      09d2a7395fa2:/fhirKeyStore.p12
    
    1. Login to Nifi – http://localhost:8080/nifi/?processGroupId=root&componentIds=1aef81c1-0172-1000-16cd-37702389d8d3

    2. Add an InvokeHTTP

      1. Click Configure
      2. Click on properties
      3. Enter Remote URL – https://host.docker.internal:9443/fhir-server/api/v4/metadata
      4. Enter Basic Authentication Username – fhiruser
      5. Enter Basic Authentication Password – change-password
      6. Click SSL Context Service
        1. Click the Drop Down
        2. Click Create Service – StandardRestrictedSSLContextService
        3. Click Create
        4. Click the Arrow to configure
        5. When prompted "Save changes before going to this Controller Service?", click Yes.
        6. Click Configure
        7. Click Properties
          1. Click Truststore Filename, and enter /fhirKeyStore.p12
          2. Click Truststore Passowrd, and enter change-password
          3. Click Truststore Type, and enter PKCS12
        8. Click Apply
        9. Check the State – Validating, you may have to refresh, until it says disabled.
        10. On the left, click enabled, and turn it on, and click enable. It may take a minute
        11. It’s basically set up, now let’s get some output.
    3. Add an LogMessage

      1. Select all Types
    4. Link the Two Nodes

    5. Click Play

    You’ll see your Nifi flow working.

    You can always use the docker image for the IBM FHIR Server https://hub.docker.com/r/ibmcom/ibm-fhir-server

  • jq fu

    Extracting a Resource from an Array

    Extracting a resource from a FHIR Bundle with over 10000 entries, and you know there is a problem at a specific resource, then you can use jq and array processing to extract the resource:

    single_patient_bundle-03-09-2020/9b3f6160-285d-4319-8d15-ac07ee3d3a8e.json \
        | jq '.entry[12672].resource'
    {
      "id": "99274e87-db14-43fa-9ada-2fcb6c1d68a6",
      "meta": {
        "profile": [
          "http://hl7.org/fhir/StructureDefinition/vitalspanel",
          "http://hl7.org/fhir/StructureDefinition/vitalsigns"
        ]
      },
      "status": "final",
      "resourceType": "Observation"
    }```

    Extracting two correlated values

    Extracting two correlated values, you can use the multiple selectors, such as the following:

    cat single_patient_bundle-03-09-2020/9b3f6160-285d-4319-8d15-ac07ee3d3a8e.json \ 
       | jq '.entry[] | "\(.status),\(.resourceType)"' | sort -u 
    final,Observation
    

    Checking the Supported Profiles on the IBM FHIR Server

    This is a handy curl to check what profiles are loaded on your IBM FHIR Server.

    Request

    curl -ks -u fhiruser:change-password https://localhost:9443/fhir-server/api/v4/metadata 2>&1 | jq -r '.rest[].resource[] | "\(.type),\(.supportedProfile)"'
    

    Processed Response

    PractitionerRole,["http://hl7.org/fhir/us/carin/StructureDefinition/carin-bb-practitionerrole|0.1.0","http://hl7.org/fhir/us/core/StructureDefinition/us-core-practitionerrole|3.1.0","http://hl7.org/fhir/us/davinci-pdex-plan-net/StructureDefinition/plannet-PractitionerRole|0.1.0"]
    Procedure,["http://hl7.org/fhir/us/core/StructureDefinition/us-core-procedure|3.1.0"]
    Provenance,["http://hl7.org/fhir/StructureDefinition/ehrsrle-provenance|4.0.1","http://hl7.org/fhir/StructureDefinition/provenance-relevant-history|4.0.1","http://hl7.org/fhir/us/core/StructureDefinition/us-core-provenance|3.1.0"]
    Questionnaire,["http://hl7.org/fhir/StructureDefinition/cqf-questionnaire|4.0.1"]
    QuestionnaireResponse,null
    RelatedPerson,["http://hl7.org/fhir/us/carin/StructureDefinition/carin-bb-relatedperson|0.1.0"]
    RequestGroup,["http://hl7.org/fhir/StructureDefinition/cdshooksrequestgroup|4.0.1"]
    ResearchDefinition,null
    ResearchElementDefinition,null
    ResearchStudy,null
    ResearchSubject,null
    

    Extracting Search Parameters with a Type Composite

    cat ./fhir-registry/definitions/search-parameters.json | jq -r '.entry[].resource | select(.type == "composite") | .expression' | sort -u

    ActivityDefinition.useContext
    CapabilityStatement.useContext | CodeSystem.useContext | CompartmentDefinition.useContext | ConceptMap.useContext | GraphDefinition.useContext | ImplementationGuide.useContext | MessageDefinition.useContext | NamingSystem.useContext | OperationDefinition.useContext | SearchParameter.useContext | StructureDefinition.useContext | StructureMap.useContext | TerminologyCapabilities.useContext | ValueSet.useContext
    ChargeItemDefinition.useContext
    DocumentReference.relatesTo
    EffectEvidenceSynthesis.useContext
    EventDefinition.useContext
    Evidence.useContext
    EvidenceVariable.useContext
    ExampleScenario.useContext
    Group.characteristic
    Library.useContext
    Measure.useContext
    MolecularSequence.referenceSeq
    MolecularSequence.variant
    Observation
    Observation | Observation.component
    Observation.component
    PlanDefinition.useContext
    Questionnaire.useContext
    ResearchDefinition.useContext
    ResearchElementDefinition.useContext
    RiskEvidenceSynthesis.useContext
    TestScript.useContext
    

    Extracting Composite Codes from Search Parameters

    cat ./fhir-registry/definitions/search-parameters.json | jq -r '.entry[].resource | select(.type == "composite") | .code'

    context-type-quantity
    context-type-value
    context-type-quantity
    context-type-value
    context-type-quantity
    context-type-value
    relationship
    ...
    chromosome-variant-coordinate
    chromosome-window-coordinate
    referenceseqid-variant-coordinate
    referenceseqid-window-coordinate
    code-value-concept
    code-value-date
    code-value-quantity
    code-value-string
    combo-code-value-concept
    combo-code-value-quantity
    component-code-value-concept
    component-code-value-quantity
    ...
    context-type-quantity
    context-type-value
    

    Handy Command to get Duplicate Search Parameters

  • Fun with Patent Data: Thomas Edison Jupyter Notebook

    Thomas Alva Edison was a famous American inventor and businessman, “described as America’s greatest inventor”, and was one of the most prolific inventors in US history. Thomas Edison was granted/filed 1084 patents from 1847-1931.[1] He’s just one cool inventor – lamps, light bulbs, phonograph and so many more life changing inventions.

    Google Patents has a wonderful depth of patent history, and the history is searchable with custom search strings:

    • inventor:(Thomas Edison) before:priority:19310101
    • inventor:(Paul R Bastide) after:priority:2009-01-01

    Google provides a seriously cool feature – a downloadable csv. Pandas anyone? The content is provided in an agreement between the USPTO and Google. Google also provides it as part of the Google APIs/Platform. The data is fundamentally public, and Google has made it very accessible with some GitHub examples. [2] The older patent data more difficult to search as the content has been scraped from Optical Character Recognition.

    I have found a cross-section of three things I am very interested in: History, Inventing and Data Science. Time to see what cool things about the Edison data.

    Step

    To start the playing with the data, one must install Jupyter.

    python3 -m pip install --upgrade pip
    python3 -m pip install jupyter

    Launch jupyter and navigate to the http://localhost:8888/tree

    jupyter notebook

    Load and Launch the notebook

    1. Download the Edison.ipynb
    2. Unzip the Edison.ipynb.zip
    3. Upload the Edison.ipynb to Jupyter
    4. Launch the Edison notebook and follow along with the cells.

    The notebook renders some interesting insights using numpy, pandas, matplotlib and scipy. The notebook includes a cell to install python libraries, and once one executes the per-requisites cell; all is loaded.

    The Jupyter notebook loads the data using an input cell, once run, the analytics enable me to see the number of co-inventors (but need to cleanse the data first).

    One notices that Thomas Alva is not an inventor in those results, as such one needs to modify to the notebook to use the API with more recent Inventors. With the comprehensive APIs from USPTO, one extracts patent data by one of a number of JSON REST APIs. Kudos to the USPTO to really open up the data and the API.

    Conclusion

    All-in the APIs/Python/Jupyter Notebook/Analysis are for fun, and provide insight into Thomas Edison’s patent data – one focused individual.

    References

    [1] Prolific Inventors https://en.wikipedia.org/wiki/List_of_prolific_inventors number wise it appears to conflict with https://en.wikipedia.org/wiki/List_of_Edison_patents which reports 1093 (it’s inclusive of design patents)
    [2] Google / USPTO Patent Data https://www.google.com/googlebooks/uspto-patents.html
    [3] USPTO Open Data https://developer.uspto.gov/about-open-data and https://developer.uspto.gov/api-catalog
    [4] PatentsView http://www.patentsview.org/api/faqs.html

  • AppDev: Zookeeper Port Forwarding to all servers from local machine

    To simply testing with Zookeeper on a remote Kafka cluster, one must connect to the client application ports on the backend.  When the remote Kafka cluster has multiple nodes and behind a firewall and a SSH jump server, the complexity is fairly high.  Note, the SSH jump server is the permitted man in the middle.    The client must allow application access to Zookeeper on Kafka – listening locally. Current techniques allow for a single port hosted on the developers machine for instance, 2181 listening on the local machine, and a single remote server.  This approach is not reliable – servers are taken out of service, added back, fail, or reroute to the master (another separate server).   

    PortDescription
    88/tcpKerberos
    2181/tcpzookeeper.property.clientPort

    A typical connection looks like: 

    ssh -J jump-server kafka-1 -L 2181:kafka-1:2181 "while true; do echo "waiting"; sleep 180; done"

      I worked to develop a small proxy. Setup hosts file. 1 – Edit /etc/hosts 2 – Add entry to hosts file

    127.0.0.1 kafka-1
    127.0.0.2 kafka-2
    127.0.0.3 kafka-3
    127.0.0.4 kafka-4
    127.0.0.5 kafka-5

    3 – Save the hosts file 4 – Setup Available interfaces (1 for each unique service) 1 is already up and in use (you only need to add the extras)

    sudo ifconfig lo0 alias 127.0.0.2 up
    sudo ifconfig lo0 alias 127.0.0.3 up
    sudo ifconfig lo0 alias 127.0.0.4 up
    sudo ifconfig lo0 alias 127.0.0.5 up

    5 – Setup the port forwarding, forward to jump server ssh -L 30991:localhost:30991 jump-server 6 – Forward to Kafka server ssh -L 30991:localhost:2181 kafka-1 7 – Loop while on kafka server while true; do echo “waiting”; sleep 180; done 8 – Repeat for each kafka server increasing the port by 1 (refer to ports section for mapping) 9 – Setup the Terminal – node krb5-tcp.js 10 – Setup the Terminal – node proxy_socket.js

    echo stats | nc kafka-1 2181
    Zookeeper version: 3.4.6-IBM_4–1, built on 06/17/2016 01:58 GMT
    Clients:
    /192.168.12.47:50404[1](queued=0,recved=1340009,sent=1360508)
    /192.168.12.46:48694[1](queued=0,recved=1348346,sent=1368936)
    /192.168.12.48:39842[1](queued=0,recved=1341655,sent=1362178)
    /0:0:0:0:0:0:0:1:39644[0](queued=0,recved=1,sent=0)

    Latency min/avg/max: 0/0/2205
    Received: 4878752
    Sent: 4944171
    Connections: 4
    Outstanding: 0
    Zxid: 0x1830001944e
    Mode: follower
    Node count: 442

    11 – Use your code to access Zookeeper ServerReferences https://github.com/nodejitsu/node-http-proxy

    sudo ifconfig lo0 alias 127.0.0.6 up
    sudo ifconfig lo0 alias 127.0.0.7 up
    sudo ifconfig lo0 alias 127.0.0.8 up

    Configuration

    {

    "2181": {

    "type": "socket",

    "members": [

    { "hostname": "kafka-1", "port": 30991 },

    { "hostname": "kafka-2", "port": 30992 },

    { "hostname": "kafka-3", "port": 30993 },

    { "hostname": "kafka-4", "port": 30994 },

    { "hostname": "kafka-5", "port": 30995 }

    ]

    }

    }
    Jaas Configuration
    ./kerberos/src/main/java/demo/kerberos/jaas.conf
    TestClient {
    com.sun.security.auth.module.Krb5LoginModule required
    principal="ctest4@test.COM"
    debug=true
    useKeyTab=true
    storeKey=true
    doNotPrompt=false
    keyTab="/Users/paulbastide/tmp/kerberos/test.headless.keytab"
    useTicketCache=false;
    };

    Java – App.java

    package demo.kerberos;

    import javax.security.auth.*;
    import javax.security.auth.login.*;
    import javax.security.auth.callback.*;
    import javax.security.auth.kerberos.*;
    import java.io.*;

    public class App {
    public static void main(String[] args) {

    System.setProperty("java.security.auth.login.config",
    "/Users/paulbastide/tmp/kerberos/src/main/java/demo/kerberos/jaas.conf");
    System.setProperty("java.security.krb5.conf", "/Users/paulbastide/tmp/kerberos/krb5.conf");

    Subject mysubject = new Subject();
    LoginContext lc;

    try {

    lc = new LoginContext("TestClient", mysubject, new MyCallBackHandler());
    lc.login();

    } catch (LoginException e) {
    e.printStackTrace();
    }

    }

    }

    Java - MyCallBackHandler.java
    package demo.kerberos;

    import javax.security.auth.*;
    import javax.security.auth.login.*;
    import javax.security.auth.callback.*;
    import javax.security.auth.kerberos.*;
    import java.io.*;

    public class MyCallBackHandler implements CallbackHandler {
    public void handle(Callback[] callbacks)
    throws IOException, UnsupportedCallbackException {

    for (int i = 0; i < callbacks.length; i++) {
    System.out.println(callbacks[i]);
    }
    }
    }
  • AppDev: Forwarding DGram in node.js

    For a project I am working on I needed to rewrite a DGram port. I moved the ports around and found a few quick tests.

    Testing with NC

    my-machine:~$ echo -n “data-message” | nc -v -4u -w1 localhost 88
    found 0 associations
    found 1 connections:
    1: flags=82<CONNECTED,PREFERRED>
    outif (null)
    src 127.0.0.1 port 53862
    dst 127.0.0.1 port 88
    rank info not available
    Connection to localhost port 88 [udp/radan-http] succeeded!
    

    Rewriting incoming datagrams to another port

    You can run the sample, and get the results as follows

    server listening 0.0.0.0:88
    server got: j��0����
    
  • Testing: Dynamic Test-NG Tests

    In my last few projects, I have used Test-NG.  Uniquely in my current project, I had to generate tests programmatically.  Instead of writing one test for each element in the project, I am able to generate a bunch at-will using the following pattern:

    Factory

    package test;
    
    import org.testng.annotations.Factory;
    
    public class DynamicTestFactory {
    
    
        @Factory
        public Object[] createInstances() {
            
         Object[] result = new Object[10]; 
         for (int i = 0; i < 10; i++) {
            result[i] = new ExampleProcessorTest(Integer.toString(i * 10)+ "A",Integer.toString(i*10) + "B");
          }
          return result;
        }
        
    }

    Test

    
    package test;
    
    import static org.testng.Assert.assertTrue;
    
    import org.testng.annotations.Test;
     
    public class ExampleProcessorTest {
    
        private String a; 
        private String b; 
        
        public ExampleProcessorTest(String a, String b) {
            this.a = a;
            this.b = b;
            
        }
        
        @Test
        public void testServer() {
            System.out.println("TEST");
            assertTrue(true);;
        }
    }

    Running Code

    [RemoteTestNG] detected TestNG version 6.9.10
    [TestNG] Running:
    /private/var/folders/07/sw3n5r3170q202d5j4tx8fhw0000gn/T/testng-eclipse--1754065412/testng-customsuite.xml
    
    TEST
    TEST
    TEST
    TEST
    TEST
    TEST
    TEST
    TEST
    TEST
    TEST
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    PASSED: testServer
    
    ===============================================
    Default test
    Tests run: 10, Failures: 0, Skips: 0
    ===============================================
    
    
    ===============================================
    Default suite
    Total tests run: 10, Failures: 0, Skips: 0
    ===============================================
    
    [TestNG] Time taken by org.testng.reporters.EmailableReporter2@76a4d6c: 9 ms
    [TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 5 ms
    [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@32cf48b7: 4 ms
    [TestNG] Time taken by org.testng.reporters.jq.Main@130f889: 23 ms
    [TestNG] Time taken by org.testng.reporters.XMLReporter@6e2c9341: 8 ms
    [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@58a90037: 10 ms
    I can also trigger using testng.xml
    <class name="DynamicTestFactory" />

    Reference

  • Release 4.1.0 – Lessons Learned in Development

    Using IBM Db2 with IBM FHIR Server

    I spent a lot of time documenting the multi-tenancy schema and how the migration works in Db2.

    Some of the critical links, the team updated, and I authored are:

    Interestingly, I spent the most time documenting the schema and how it supports the multi-tenancy with the following pattern:

    Using ibm/db2 docker and using with IBM FHIR Server

    Use the following Dockerfile from this folder fhir-install/docker.

    docker build -t fhirdb Dockerfile-db2 --squash
    

    If you create the Db2 container without the above Dockerfile, please be sure to do the following:

    1. Add the trial license.

    /opt/ibm/db2/V11.5/adm/db2licm -a /var/db2/db2trial.lic

    1. Update the Instance Memory Configuration
    su - db2inst1 -c "db2 update dbm cfg using INSTANCE_MEMORY AUTOMATIC"
    
    1. Create the Db2 database with 32K pagesize
    su - db2inst1 -c "db2 CREATE DB FHIRDB using codeset UTF-8 territory us PAGESIZE 32768"
    
    1. Add a Db2 user
    groupadd -g 1002 fhir && \
        useradd -u 1002 -g fhir -M -d /database/config/fhirserver fhirserver && \
        echo "change-password" | passwd --stdin fhirserver
    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \"grant connect on database TO USER fhirserver\""
    
    1. Run the Schema creation with a small pool
    –prop-file fhir-persistence-schema/db.properties
    –schema-name FHIRDATA
    –pool-size 2
    –update-schema
    
    1. Create a new tenant
    –prop-file fhir-persistence-schema/db.properties
    –schema-name FHIRDATA
    –allocate-tenant TNT1
    
    1. Contents of db.properties are
    db.host=localhost
    db.port=50000
    db.database=fhirdb
    user=db2inst1
    password=change-me
    sslConnection=false
    

    Good luck with Docker/IBM FHIR Server

  • Release 4.0.1 – Lessons Learned in Development

    My team just released IBM FHIR Server 4.0.1.

    A few things, I learned:

    To Reset a Tag

    git tag -d 4.0.1
    git push origin --delete 4.0.1
    git tag 4.0.1
    git push --tags
    

    End to End Windows Tests

    I used a powershell and added an archive step.

    # Create the Zip File
    $compress = @{
    LiteralPath= $pre_it_logs
    CompressionLevel = 'Fastest'
    DestinationPath = $zip_file
    }
    
    Compress-Archive @compress -Force
    

    Ignoring Self-Signed Certs with Powershell

    Also a step to test our server, but hit an issue with the certificate
    
    # Ignore Self Signed Certificates and use TLS
    # Reference https://stackoverflow.com/a/46254549/1873438
    $AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
    [System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
    Add-Type -TypeDefinition @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
    ServicePoint srvPoint, X509Certificate certificate,
    WebRequest request, int certificateProblem) {
    return true;
    }
    }
    "@
    $AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
    [System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
    [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
    

    Break it down into individual parts

    Release Tricks and Tips break it down into different workflows.

  • Gatsby & Carbon: Build with Github Action

    As some of you know, I work on the IBM FHIR Server and with my colleagues, I have started automating some of the actions we take – Build, Test, Deploy, Deploy our website.

    More specific to the “Deploy our website” automation, our website uses technologies, such as Gatsby, Carbon, Gatsby Carbon Theme. Fundamentally, a static site generation technology, like Jekyll, Gatsby uses Node, Yarn and some nice React code.

    To build our site with GitHub actions, I built out a site workflow.  The key elements to this workflow are:

    • Triggers
    • Node.js and Ubuntu Images
    • Build
    • Add, Commit and Push to GH Pages
    • Debugging and Replicating Locally

    Triggers

    For the Triggers, I recommend limiting the site generation to master branches.  The master branch filter and on push, limits the re-deployment, also keep your site building on on docs/** changes.

    on:
    push:
    paths:
    – “docs/**”
    branches:
    – master
     
    There is a subtlety the websites are cached for 10 minutes, confirmed on the site – Caching assets in website served from GitHub pages

    Node.js and Ubuntu Images

    I opted to use Ubuntu with Node.js
     
    jobs:
    build:
    runs-on: ubuntu-latest

    strategy:
    matrix:
    node-version: [12.x]
     
    The important thing is ubuntu-latest which has some incompatibility with Gatsby Carbon’s build. 

    Build

    I build the system as follows:

    Checkout the repo to a folder

    steps:
    -name: Grab the Master Branch
    uses: actions/checkout@v1
    with:
    working-directory: fhir
    ref: refs/heads/master
    fetch-depth: 1
    path: fhir
     
    Activate Node
    name: Use Node.js ${{ matrix.node-version }}
    uses: actions/setup-node@v1
    with:
    node-version: ${{ matrix.node-version }}
     
    Setup the build
     
    echo “Check on Path”
    pwd
    cd docs/
    npm install -g gatsby-cli
    gatsby telemetry –disable
     
    Install the packages, note, fsevents is not used on linux images, so use–no-optional (these plugins are suspect).
     
    npm install –no-optional –save react react-copy-to-clipboard react-dom react-ga classnames carbon @carbon/addons-website carbon-components carbon-components-react carbon-addons-cloud carbon-icons gatsby gatsby-theme-carbon-starter markdown-it gatsby-plugin-manifest gatsby-plugin-slug gatsby-plugin-sitemap gatsby-plugin-sharp
     
    With ubuntu, you can’t use gatsby build directly per https://github.com/gatsbyjs/gatsby/issues/17557, 
    so I use the suggestion as a workaround due to path/issues in the gatsby component dependency of fsevents.
     
    npm –prefix-paths run build
    cp -R public/ ../../public/
     
    Grab the GH-Pages branch
     
    – name: Grab the GH Pages Branch
    uses: actions/checkout@v1
    with:
    working-directory: gh-pages
    ref: refs/heads/gh-pages
    fetch-depth: 1
    path: docs
    token: ${{ secrets.GITHUB_TOKEN }}

    Per Bypassing Jekyll on GitHub Pages, be sure to add the .nojekll to the root of the gh-pages. I added a guard in the shell script to check if the file is there, and create the file if it does not exist.

    If you need Environment variables, you should add the environment variables to the step.

    Add, Commit and Push to GH Pages

    I add the gitignore and nojekll files while removing any cached files, before moving in the new files.

    I also like to make sure when this runs there is a build.txt file to trace when the site is built. (This file contains the build time Thu Nov 21 19:39:49 UTC 2019

    I then use the github environment variables passed in to push the contents to the repo the branch is from. 

    -name: Commit and Add GH Pages

    run: |

    echo "cleaning up the prior files on the branch"

    if [ ! -f .nojekyll ]

    then

    touch .nojekyll

    rm -f _config.yml

    fi

    rm -f *.js webpack.stats.json styles-*.js styles-*.js.map webpack-runtime-*.js.map webpack-runtime-*.js manifest.webmanifest component---*.js* app-*.js*

    rm -rf docs/node_modules docs/public docs/.cache

    echo "Moving the files around for gh-pages"

    cp -Rf ../public/* ./

    find .

    date > build.txt

    git config --global user.email "${{ secrets.GITHUB_ACTOR }}@users.noreply.github.com"

    git config --global user.name "Git Hub Site Automation"

    git add .

    git commit -m "Update to GH-Pages"

    - name: Push changes to GH Pages

    run: |

    echo "Push Changes"

    git branch

    remote_repo="https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"

    git push "${remote_repo}" HEAD:gh-pages

    env:

    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

    GITHUB_REPOSITORY: ${{ secrets.GITHUB_REPOSITORY }}

    GITHUB_ACTOR: ${{ secrets.GITHUB_ACTOR }}

    CI: true

    Debugging and Replicating Locally

    If you are troubleshooting, you can use a couple of approaches: 

    1 – Create a Docker Image

    Create the Image

    docker run -itd –name gatsby-node -v docs:/data node:latest

    Copy the Files

    docker cp ~/FHIR/docs 6d810efb3b586739932166d424641003ee9b238de506543fcdd47eb7e7d41699:/data

    Launch the shell and try the build

    npm install –no-optional –save react react-copy-to-clipboard react-dom react-ga classnames carbon @carbon/addons-website carbon-components carbon-components-react carbon-addons-cloud carbon-icons gatsby gatsby-theme-carbon-starter markdown-it gatsby-plugin-manifest gatsby-plugin-slug gatsby-plugin-sitemap gatsby-plugin-sharp

    Run the gatsby build

    npm –prefix-paths run build

    2. If you want complicated navigation, refer to https://github.com/gatsbyjs/gatsby/blob/master/www/src/data/sidebars/doc-links.yaml however… gatsby-carbon-theme’s sidebar links only uses the to value not the href value.

    3.  If you have an issue with your deployment check a couple of things:

    Check your Deployed Environment. You should see a deployment in the last few seconds.

    Check your Settings You should see no issues, else investigate the site locally on the gh-pages branch, and check Troubleshooting Jekyll build errors for GitHub Pages sites

    Best of luck with your build!