Category: IBM FHIR Server

  • Question: Can I validate a batch of FHIR resources?

    Yes, you can use the Batch API.

    1. The Bundle.entry must include a request object pointing at URL <RESOURCE>/$validate with method POST.

    2. The Resource that is validated must be wrapped in a Parameters object and added to the Parameters.parameter.resource

    {
        "resourceType": "Parameters",
        "parameter": [
            {
                "name": "resource",
                "resource": <COPY HERE>
            }
        ]
    }
    
    1. Embed it at Bundle.entry.resource, and make a request to the server root.
    curl --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
    --header 'Authorization: Basic ...' \
    --header 'Content-Type: application/json' \
    --data-raw '{
        "resourceType": "Bundle",
        "type": "transaction",
        "entry": [
            {
                "fullUrl": "Patient/$validate",
                "resource": {
                    "resourceType": "Parameters",
                    "parameter": [
                        {
                            "name": "resource",
                            "resource": {
                                "resourceType": "Patient",
                                "id": "example",
                                "name": [
                                    {
                                        "use": "official",
                                        "family": "Chalmers",
                                        "given": [
                                            "Peter",
                                            "James"
                                        ]
                                    }
                                ]
                            }
                        }
                    ],
                    "meta": {
                        "tag": [
                            {
                                "system": "http://terminology.hl7.org/CodeSystem/v3-ActReason",
                                "code": "HTEST",
                                "display": "test health data"
                            }
                        ]
                    }
                },
                "request": {
                    "method": "POST",
                    "url": "Patient/$validate"
                }
            }
        ]
    }'
    
    1. You’ll get a response like this, note the HTTP Status Code is 200 coming back… You have to look at Bundle.entry.resource.response.issue to check the severity and details to see why the validation is failing.
    {
        "resourceType": "Bundle",
        "type": "transaction-response",
        "entry": [
            {
                "resource": {
                    "resourceType": "OperationOutcome",
                    "id": "NoError",
                    "text": {
                        "status": "additional",
                        "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>No error</p></div>"
                    },
                    "issue": [
                        {
                            "severity": "warning",
                            "code": "invariant",
                            "details": {
                                "text": "dom-6: A resource should have narrative for robust management"
                            },
                            "expression": [
                                "Patient"
                            ]
                        }
                    ]
                },
                "response": {
                    "status": "200",
                    "outcome": {
                        "resourceType": "OperationOutcome",
                        "id": "NoError",
                        "text": {
                            "status": "additional",
                            "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>No error</p></div>"
                        },
                        "issue": [
                            {
                                "severity": "warning",
                                "code": "invariant",
                                "details": {
                                    "text": "dom-6: A resource should have narrative for robust management"
                                },
                                "expression": [
                                    "Patient"
                                ]
                            }
                        ]
                    }
                }
            }
        ]
    }
    

    You should add entries for each Resource you want to validate in the Batch.

  • Docker Compose and Not Able to set a TTY in a Git Hub actions

    In my GitHub Repository IBM FHIR Server, we use GitHub Actions to execute our Continuous Integration (CI) workflows. The CI workflows enable us to execute our code in complicated scenarios – a database (IBM Db2, Postgres), events system (Kafka (two zookeeper, two brokers), NATS) in a mix of various configurations.

    I started to enable Docker Compose with Kafka and the IBM FHIR Server’s Audit module.

    I ran into this error when running docker-compose exec in my workflow’s run.

    TEST_CONFIGURATION: check that there is output and the configuration works
    the input device is not a TTY
    Error: Process completed with exit code 1.
    

    It turns out this is well known in GitHub Actions https://github.com/actions/runner/issues/241#issuecomment-745902718 with a great example of a fix at https://github.com/gfx/example-github-actions-with-tty/blob/4c5f457c65dfe61e273e0470414699420be5e134/.github/workflows/test.yml#L18

    I ended up changing my workflow, and it passed!

    The key being in setting the shell value – shell: 'script -q -e -c "bash {0}"'

    - name: Server Integration Tests - Audit
    env:
    WORKSPACE: ${{ github.workspace }}
    shell: 'script -q -e -c "bash {0}"'
    run: |
    bash build/audit/bin/pre-integration-test.sh ${{matrix.audit}}
    bash build/audit/bin/integration-test.sh ${{matrix.audit}}
    bash build/audit/bin/post-integration-test.sh ${{matrix.audit}}
    
  • IBM FHIR Server – Using the Docker Image with Near Feature and FHIR Examples from Jupyter Notebooks

    Hi Everyone.

    Thanks for sitting down and watching this video. I’m going to show you how to quickly spin up a Docker image of IBM FHIR Server, check the logs, make sure it’s healthy, and how to use the fhir-examples module with the near search.

    The following are the directions followed in the video:

    Navigate to DockerHub: IBM FHIR Server

    Run the Server docker run -p9443:9443 ibmcom/ibm-fhir-server

    Note, startup may take 2 minutes as the image is bootstrapping a new Apache Derby database in the image. To use Postgres or IBM Db2, please review the documentation.

    Review the docker logs

    Check the server is up and operational curl -k -i -u 'fhiruser:change-password' 'https://localhost:9443/fhir-server/api/v4/$healthcheck'

    You now have a running IBM FHIR Sever.

    Let’s load some data using a Jupyter Notebook.

    The IBM FHIR Server team wraps specification and service unit tests into a module called fhir-examples and posts to Bintray: ibm-fhir-server-releases or go directly to the repository.

    We’re going to use the python features and Jupyter Notebook to process the fhir-examples.

    We’ll download the zip, filter the interesting jsons, and upload to the IBM FHIR Server in a loop.

    entries = z.namelist()
    for entry in entries:
    if entry.startswith('json/ibm/bulk-data/location/'):
    f = z.open(entry);
    content = f.read()
    r = requests.post('https://localhost:9443/fhir-server/api/v4/Location',
    data=content,
    headers=headers,
    auth=httpAuth,
    verify=False)
    print('Done uploading - ' + entry)
    

    We’re going to query the data on the IBM FHIR Server using the Search Query Parameter near to search within 10Km of Cambridge Massachusetts.

    queryParams = {
    'near': '42.373611|-71.110558|10|km',
    "_count" : 200
    }
    

    Note, the IBM FHIR Server includes some additional search beyond the UCUM and WS48 units and it’s listed in at the Conformance page.

    We’ll normalize this data and put in a Pandas dataframe.

    From the dataframe, we can now add markers to the page.

    cambridge = [ 42.373611, -71.11000]
    map_cambridge_locs_from_server = folium.Map(location=cambridge, zoom_start=10)
    
    # Iterate through the Rows
    for location_row in location_rows :
    # print(location_row)
    # Cast the values into the appropriate types as FOLIUM will die weirdly without it.
    lat_inc = float(location_row['resource.position.latitude'])
    long_inc = float(location_row['resource.position.longitude'])
    name_inc = str(location_row['resource.name'])
    #print(lat_inc)
    #print(long_inc)
    #print(name_inc)
    label = folium.Popup(name_inc, parse_html=True)
    folium.CircleMarker(
    [lat_inc, long_inc],
    radius=5,
    popup=label,
    fill=True,
    fill_color='red',
    fill_opacity=0.7).add_to(map_cambridge_locs_from_server)
    map_cambridge_locs_from_server
    

    You can see the possibilities with the IBM FHIR Server and the near search.

    Reference

  • jq fu for FHIR Capability Statements

    The following takes the Capability Statements and generates the minimum coverage set of Resources across implementation guides based on the capability statements.

    Resources and Profiles output in Markdown

    This is handy to check the number of profiles in use (beyond the base spec) and to know the profiles used on the server based on the capability statements.

    Call

    cat capabilitystatements/*.json | jq -r '.rest[].resource[]| "|\(.type)|\(.supportedProfile)|"' | sort -u

    Output

    Resource Profiles
    AllergyIntolerance http://hl7.org/fhir/us/core/StructureDefinition/us-core-allergyintolerance
    CarePlan http://hl7.org/fhir/us/core/StructureDefinition/us-core-careplan
    CareTeam http://hl7.org/fhir/us/core/StructureDefinition/us-core-careteam
    Condition http://hl7.org/fhir/us/core/StructureDefinition/us-core-condition
    Device http://hl7.org/fhir/us/core/StructureDefinition/us-core-implantable-device

    Resource and Operations

    To check the operations rrequired in the implementation guides, you can use the following to process a set of capability statements into a useful outputs.

    Call

    cat capabilitystatements/*.json | jq -r '.rest[].resource[]| "|\(.type)|\(.operation)|"' | grep -v null
    

    Output

    Resource Operation Conformance
    ValueSet $expand SHOULD

    I hope this helps.

  • Release 4.2.2 – Notes

    My Team just released IBM FHIR Server 4.2.2. Other than the amazing things documented and released with the release tab, I learned a few things.

    Replace Tags

    If you need to replace tags, force it with fetch

    ~$ git fetch --tags -f
    From github.com:IBM/FHIR
    t [tag update] 4.1.0 -> 4.1.0
    t [tag update] 4.2.2 -> 4.2.2
    

    Rebuild the Validation Package

    export BUILD_TYPE=RELEASE
    export BUILD_VERSION=4.2.2
    bash build/release/version.sh
    
    mvn ${THREAD_COUNT} -ntp -B clean source:jar source:test-jar javadoc:jar \
    install -f fhir-parent -Pfhir-validation-distribution,fhir-ig-carin-\
    bb,fhir-ig-davinci-pdex-plan-net,fhir-ig-mcode,fhir-ig-us-core,deploy-\
    bintray -DskipTests -pl ../fhir-ig-davinci-pdex-plan-net/,../fhir-\
    validation -amd
    

    -amd keeps the build focused only on the necessary packages (not the full fhir-parent)

    Idempotent Execution of the Role Creation

    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRSERVER') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRSERVER'; END IF; END;\""

    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRBATCH') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRBATCH'; END IF; END;\""

    Shell Pipestatus

    Checking the Status of any command in a pipe, it was helpful in some automation where I had to wait on a jar to finish and check the output. Source

    Command

    curl -L https://google.com | grep response | tee response.txt
    RC=${PIPESTATUS[1]}
    echo $RC
    

    Output

    4
    

    Reference

  • Apache Nifi and IBM FHIR Server: InvokeHTTP and SSL

    A user who is integrating Apache Nifi and IBM FHIR Server asked how they get the SSL to work between the two, and here is a small recipe for you:

    1. List Keys
    keytool -list -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password -rfc
    

    Check to see if you have a default, if you do, go to step 2, else step 3.

    1. Change default
    keytool -changealias -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
       -storepass change-password -alias default -destalias old_default
    

    You can always double check with step 3.

    1. Create a new default with a distinguished name for your hostname (mine is host.docker.internal)
    keytool -genkey -keyalg RSA -alias default -keystore \
    fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password -validity 2000 -keysize 2048 -dname cn=host.docker.internal
    
    1. Confirm the lists of keys
    keytool -list -keystore \
      fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      -storepass change-password
    
    Keystore type: PKCS12
    Keystore provider: SUN
    
    Your keystore contains 2 entries
    
    old_default, May 15, 2020, PrivateKeyEntry,
    Certificate fingerprint (SHA-256): 9D:94:C2:F8:C1:51:9B:0F:21:50:4F:BB:60:A4:8A:3F:AF:C0:F0:13:C4:80:BE:A3:94:42:04:46:56:DB:D9:7B
    default, May 15, 2020, PrivateKeyEntry,
    Certificate fingerprint (SHA-256): 5B:38:D5:FD:7F:8A:80:60:12:CF:7F:61:C6:D6:C5:54:F3:FD:F8:80:34:58:A5:3F:1C:8F:2C:0A:42:85:C0:49
    

    Notice, the new key.

    1. Restart your app server to pick up the latest. Once restarted, proceed to next step.

    2. Confirm you see the subject is the one you need.

    curl -k https://localhost:9443 -v 2>&1 | grep -i subject
    *  subject: CN=host.docker.internal
    
    1. Start a nifi image
    docker run -p 8080:8080 --rm apache/nifi:latest bash
    
    1. Find the docker container id
    $ docker ps
    CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                                                   NAMES
    09d2a7395fa2        apache/nifi:latest   "../scripts/start.sh…"   7 seconds ago       Up 6 seconds        8000/tcp, 8443/tcp, 10000/tcp, 0.0.0.0:8080->8080/tcp   gracious_rosalind
    
    1. Copy the fhirKeystore.p12 (in this case we just updated this one only).
    docker cp fhir-server-dist/wlp/usr/servers/fhir-server/resources/security/fhirKeyStore.p12 \
      09d2a7395fa2:/fhirKeyStore.p12
    
    1. Login to Nifi – http://localhost:8080/nifi/?processGroupId=root&componentIds=1aef81c1-0172-1000-16cd-37702389d8d3

    2. Add an InvokeHTTP

      1. Click Configure
      2. Click on properties
      3. Enter Remote URL – https://host.docker.internal:9443/fhir-server/api/v4/metadata
      4. Enter Basic Authentication Username – fhiruser
      5. Enter Basic Authentication Password – change-password
      6. Click SSL Context Service
        1. Click the Drop Down
        2. Click Create Service – StandardRestrictedSSLContextService
        3. Click Create
        4. Click the Arrow to configure
        5. When prompted "Save changes before going to this Controller Service?", click Yes.
        6. Click Configure
        7. Click Properties
          1. Click Truststore Filename, and enter /fhirKeyStore.p12
          2. Click Truststore Passowrd, and enter change-password
          3. Click Truststore Type, and enter PKCS12
        8. Click Apply
        9. Check the State – Validating, you may have to refresh, until it says disabled.
        10. On the left, click enabled, and turn it on, and click enable. It may take a minute
        11. It’s basically set up, now let’s get some output.
    3. Add an LogMessage

      1. Select all Types
    4. Link the Two Nodes

    5. Click Play

    You’ll see your Nifi flow working.

    You can always use the docker image for the IBM FHIR Server https://hub.docker.com/r/ibmcom/ibm-fhir-server

  • jq fu

    Extracting a Resource from an Array

    Extracting a resource from a FHIR Bundle with over 10000 entries, and you know there is a problem at a specific resource, then you can use jq and array processing to extract the resource:

    single_patient_bundle-03-09-2020/9b3f6160-285d-4319-8d15-ac07ee3d3a8e.json \
        | jq '.entry[12672].resource'
    {
      "id": "99274e87-db14-43fa-9ada-2fcb6c1d68a6",
      "meta": {
        "profile": [
          "http://hl7.org/fhir/StructureDefinition/vitalspanel",
          "http://hl7.org/fhir/StructureDefinition/vitalsigns"
        ]
      },
      "status": "final",
      "resourceType": "Observation"
    }```

    Extracting two correlated values

    Extracting two correlated values, you can use the multiple selectors, such as the following:

    cat single_patient_bundle-03-09-2020/9b3f6160-285d-4319-8d15-ac07ee3d3a8e.json \ 
       | jq '.entry[] | "\(.status),\(.resourceType)"' | sort -u 
    final,Observation
    

    Checking the Supported Profiles on the IBM FHIR Server

    This is a handy curl to check what profiles are loaded on your IBM FHIR Server.

    Request

    curl -ks -u fhiruser:change-password https://localhost:9443/fhir-server/api/v4/metadata 2>&1 | jq -r '.rest[].resource[] | "\(.type),\(.supportedProfile)"'
    

    Processed Response

    PractitionerRole,["http://hl7.org/fhir/us/carin/StructureDefinition/carin-bb-practitionerrole|0.1.0","http://hl7.org/fhir/us/core/StructureDefinition/us-core-practitionerrole|3.1.0","http://hl7.org/fhir/us/davinci-pdex-plan-net/StructureDefinition/plannet-PractitionerRole|0.1.0"]
    Procedure,["http://hl7.org/fhir/us/core/StructureDefinition/us-core-procedure|3.1.0"]
    Provenance,["http://hl7.org/fhir/StructureDefinition/ehrsrle-provenance|4.0.1","http://hl7.org/fhir/StructureDefinition/provenance-relevant-history|4.0.1","http://hl7.org/fhir/us/core/StructureDefinition/us-core-provenance|3.1.0"]
    Questionnaire,["http://hl7.org/fhir/StructureDefinition/cqf-questionnaire|4.0.1"]
    QuestionnaireResponse,null
    RelatedPerson,["http://hl7.org/fhir/us/carin/StructureDefinition/carin-bb-relatedperson|0.1.0"]
    RequestGroup,["http://hl7.org/fhir/StructureDefinition/cdshooksrequestgroup|4.0.1"]
    ResearchDefinition,null
    ResearchElementDefinition,null
    ResearchStudy,null
    ResearchSubject,null
    

    Extracting Search Parameters with a Type Composite

    cat ./fhir-registry/definitions/search-parameters.json | jq -r '.entry[].resource | select(.type == "composite") | .expression' | sort -u

    ActivityDefinition.useContext
    CapabilityStatement.useContext | CodeSystem.useContext | CompartmentDefinition.useContext | ConceptMap.useContext | GraphDefinition.useContext | ImplementationGuide.useContext | MessageDefinition.useContext | NamingSystem.useContext | OperationDefinition.useContext | SearchParameter.useContext | StructureDefinition.useContext | StructureMap.useContext | TerminologyCapabilities.useContext | ValueSet.useContext
    ChargeItemDefinition.useContext
    DocumentReference.relatesTo
    EffectEvidenceSynthesis.useContext
    EventDefinition.useContext
    Evidence.useContext
    EvidenceVariable.useContext
    ExampleScenario.useContext
    Group.characteristic
    Library.useContext
    Measure.useContext
    MolecularSequence.referenceSeq
    MolecularSequence.variant
    Observation
    Observation | Observation.component
    Observation.component
    PlanDefinition.useContext
    Questionnaire.useContext
    ResearchDefinition.useContext
    ResearchElementDefinition.useContext
    RiskEvidenceSynthesis.useContext
    TestScript.useContext
    

    Extracting Composite Codes from Search Parameters

    cat ./fhir-registry/definitions/search-parameters.json | jq -r '.entry[].resource | select(.type == "composite") | .code'

    context-type-quantity
    context-type-value
    context-type-quantity
    context-type-value
    context-type-quantity
    context-type-value
    relationship
    ...
    chromosome-variant-coordinate
    chromosome-window-coordinate
    referenceseqid-variant-coordinate
    referenceseqid-window-coordinate
    code-value-concept
    code-value-date
    code-value-quantity
    code-value-string
    combo-code-value-concept
    combo-code-value-quantity
    component-code-value-concept
    component-code-value-quantity
    ...
    context-type-quantity
    context-type-value
    

    Handy Command to get Duplicate Search Parameters

  • Release 4.1.0 – Lessons Learned in Development

    Using IBM Db2 with IBM FHIR Server

    I spent a lot of time documenting the multi-tenancy schema and how the migration works in Db2.

    Some of the critical links, the team updated, and I authored are:

    Interestingly, I spent the most time documenting the schema and how it supports the multi-tenancy with the following pattern:

    Using ibm/db2 docker and using with IBM FHIR Server

    Use the following Dockerfile from this folder fhir-install/docker.

    docker build -t fhirdb Dockerfile-db2 --squash
    

    If you create the Db2 container without the above Dockerfile, please be sure to do the following:

    1. Add the trial license.

    /opt/ibm/db2/V11.5/adm/db2licm -a /var/db2/db2trial.lic

    1. Update the Instance Memory Configuration
    su - db2inst1 -c "db2 update dbm cfg using INSTANCE_MEMORY AUTOMATIC"
    
    1. Create the Db2 database with 32K pagesize
    su - db2inst1 -c "db2 CREATE DB FHIRDB using codeset UTF-8 territory us PAGESIZE 32768"
    
    1. Add a Db2 user
    groupadd -g 1002 fhir && \
        useradd -u 1002 -g fhir -M -d /database/config/fhirserver fhirserver && \
        echo "change-password" | passwd --stdin fhirserver
    su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \"grant connect on database TO USER fhirserver\""
    
    1. Run the Schema creation with a small pool
    –prop-file fhir-persistence-schema/db.properties
    –schema-name FHIRDATA
    –pool-size 2
    –update-schema
    
    1. Create a new tenant
    –prop-file fhir-persistence-schema/db.properties
    –schema-name FHIRDATA
    –allocate-tenant TNT1
    
    1. Contents of db.properties are
    db.host=localhost
    db.port=50000
    db.database=fhirdb
    user=db2inst1
    password=change-me
    sslConnection=false
    

    Good luck with Docker/IBM FHIR Server

  • Release 4.0.1 – Lessons Learned in Development

    My team just released IBM FHIR Server 4.0.1.

    A few things, I learned:

    To Reset a Tag

    git tag -d 4.0.1
    git push origin --delete 4.0.1
    git tag 4.0.1
    git push --tags
    

    End to End Windows Tests

    I used a powershell and added an archive step.

    # Create the Zip File
    $compress = @{
    LiteralPath= $pre_it_logs
    CompressionLevel = 'Fastest'
    DestinationPath = $zip_file
    }
    
    Compress-Archive @compress -Force
    

    Ignoring Self-Signed Certs with Powershell

    Also a step to test our server, but hit an issue with the certificate
    
    # Ignore Self Signed Certificates and use TLS
    # Reference https://stackoverflow.com/a/46254549/1873438
    $AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
    [System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
    Add-Type -TypeDefinition @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
    ServicePoint srvPoint, X509Certificate certificate,
    WebRequest request, int certificateProblem) {
    return true;
    }
    }
    "@
    $AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
    [System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
    [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
    

    Break it down into individual parts

    Release Tricks and Tips break it down into different workflows.