Category: IBM FHIR Server

IBM FHIR Server – Debugging Tips
Checking a Postgres Function Definition

I needed to verify my postgres function.
1. Shows the contents of the functions in the schema.
```
SELECT pg_get_functiondef(f.oid)
FROM pg_catalog.pg_proc f
INNER JOIN pg_catalog.pg_namespace n ON (f.pronamespace = n.oid)
WHERE n.nspname = 'fhirdata';
```
1. Show all the details of the functions in the schema.
```
SELECT *
FROM pg_catalog.pg_proc f
INNER JOIN pg_catalog.pg_namespace n ON (f.pronamespace = n.oid)
WHERE n.nspname = 'fhirdata';
```
Tracing Bulk Data with Cloud Object Storage

I wanted to figure out why my code was failing to connect to the backend S3 bucket.

I used the environment variable in my docker image called TRACE_SPEC. TRACE_SPEC is loaded into the logging as the traceSpecification.

I set this to *=info:com.ibm.cloud.*=FINEST which spits out great detail to S3 using the IBM COS SDK.

The output looks like:

With this level of trace, you can really dive into the connection, and determine what is going on.

Note, if you want the whole picture of what is happening with COS and JavaBatch and BulkData, use the following:

*=info:com.ibm.fhir.*=finer:RRA=all:WAS.j2c=all:com.ibm.ws.jdbc.*=all:Transaction=all:ConnLeakLogic=all:Transaction=all:com.ibm.ws.transaction.services.WebAppTransactionCollaboratorImpl=all:RRA=all:com.ibm.cloud.*=FINEST
2021-06-03

BulkData: rclone with S3

This was super helpful for debugging S3 files/folders.

brew install rclone
rclone rcd --config rclone.conf  --rc-web-gui --no-check-certificate --rc-user rc --rc-pass rc

[minio]
type = s3
env_auth = false
access_key_id = minio
secret_access_key = demooo-password
region = us-east-1
endpoint = https://localhost:9000
location_constraint = 
server_side_encryption =

rclone sync

rclone sync --config rclone-s3.conf --no-check-certificate -i fhir-performance:fhir-performance $(pwd)

2021-05-22

GitHub Actions: Paging file is too small
If you hit this issue, per the GitHub community it’s a Windows issue that can be worked around with a Pagefile. link

[ERROR] OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000700000000, 553648128, 0) failed; error='The paging file is too small for this operation to complete' (DOS error/errno=1455)

I used the discussed GitHub Action configure-pagefile

You can see the solution inline at GitHub Actions Workflow
```
    - name: Configure Pagefile
      uses: al-cheb/configure-pagefile-action@v1.2
      with:
        minimum-size: 8GB
        maximum-size: 8GB
        disk-root: "C:"
```
2021-05-12
Some examples of FHIR Path

Here are some examples I generated from a demo with the IBM FHIR Server

Extract the references from the first subject

Bundle.entry.resource.ofType(Observation).subject[0].reference.value

Output Patient/ebb16c62-cb06-4ff8-8ce8-ccb865e7a240

Extract the references from the first subject alternative

Bundle.entry.resource.ofType(Observation).subject.first().reference.value

Output Patient/ebb16c62-cb06-4ff8-8ce8-ccb865e7a240

Select the first Patient reference where the Diastolic is over 90

Bundle.entry.resource.ofType(Observation).where(component.where(code.coding.code = '8462-4').value.where( value > 90.0).exists()).subject.reference.value.first()

Output Patient/5fabfccd-254e-42af-bed2-84199a5c05f2

Select the Patient’s first name where the reference matches the logical id

Bundle.entry.resource.ofType(Patient).where(id = '" + v.split("/")[1] + "').name.given.value

Output John

2021-05-04

Let’s Go: Start-to-End with the IBM FHIR Server and Bulk Data

In IBM FHIR Server 4.6.0, IBM FHIR Server has refactored the supported Bulk Data operations that support the HL7 FHIR BulkDataAccess IG: STU1 and the Proposal for $import Operation.

To demonstrate the new features in a quick script, use the following to setup an S3 demonstration from the start-to-end.

Setup Minio
Setup IBM FHIR Server
Load Sample Data
Run $import
Run $export

Setup Minio

Pull the minio docker image
Create a local data directory for data

mkdir -p minio-data/

Download demonstration only certificates/keys

curl -L -o private.key raw.githubusercontent.com/IBM/FHIR/main/build/docker/minio/private.key

curl -L -o public.crt https://raw.githubusercontent.com/IBM/FHIR/main/build/docker/minio/public.crt

Note, these are demonstration only Private Key and Public cert pair.

Create the Minio container

export MINIO_ROOT_PASSWORD=$(openssl rand -base64 20)
docker run -d --rm -p 9000:9000 \
  --name minio \
  -h minio \
  -e "MINIO_ROOT_USER=fhir-s3-admin" \
  -e "MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}" \
  -e "MINIO_ACCESS_KEY=fhirAccessKey" \
  -e "MINIO_SECRET_KEY=${MINIO_ROOT_PASSWORD}" \
  -v $(pwd)/minio-data:/data \
  -v $(pwd)/public.crt:/root/.minio/certs/public.crt \
  -v $(pwd)/private.key:/root/.minio/certs/private.key \
  minio/minio server /data

Note, the Minio container will create local certificates.

Using your MINIO_ROOT_PASSWORD echo ${MINIO_ROOT_PASSWORD}, log in to your Minio instance.

Create a Bucket (folder), look in the lower left. Name it fhirbulkdata.

You should see the bucket.

6. Download the [Sample NDJSON](https://github.com/IBM/FHIR/blob/main/fhir-server-test/src/test/resources/testdata/import-operation/test-import.ndjson)

`curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/import-operation/test-import.ndjson -o test-import.ndjson`

We’re going to upload this file to the fhir-bulkdata bucket.

7. Click Upload File > Select the test-import.ndjson. Click Upload. Confirm you see the file in the bucket.

Setup IBM FHIR Server

Download the fhir-server-config.json

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/fhir-server-config.json -o fhir-server-config.json

Update the fhir-server-config.json fhirServer/bulkdata with the following snippet to use hmac and bulkdata. Be sure to update fhirServer/bulkdata/storageProviders/minio/accessKeyId to fhirAccessKey and fhirServer/bulkdata/storageProviders/minio/secretAccessKey to your MINIO_ROOT_PASSWORD.

    "bulkdata": {
            "enabled": true,
            "core": {
                "api": {
                    "url": "https://localhost:9443/ibm/api/batch",
                    "user": "fhiradmin",
                    "password": "change-password",
                    "truststore": "resources/security/fhirTrustStore.p12",
                    "truststorePassword": "change-password", 
                    "trustAll": true
                },
                "cos" : { 
                    "partUploadTriggerSizeMB": 10,
                    "objectSizeThresholdMB": 200,
                    "objectResourceCountThreshold": 200000,
                    "useServerTruststore": true
                },
                "file" : { 
                    "writeTriggerSizeMB": 1,
                    "sizeThresholdMB": 200,
                    "resourceCountThreshold": 200000
                },
                "pageSize": 100,
                "batchIdEncryptionKey": "change-password",
                "maxPartitions": 3, 
                "maxInputs": 5
            },
            "storageProviders": {
                "default" : {
                    "type": "file",
                    "_type": "ibm-cos|aws-s3|file|https",
                    "validBaseUrls": [],
                    "fileBase": "/output/bulkdata",
                    "disableBaseUrlValidation": true,
                    "exportPublic": true,
                    "disableOperationOutcomes": true,
                    "duplicationCheck": false, 
                    "validateResources": false
                },
                "minio" : {
                    "type": "aws-s3",
                    "bucketName": "fhirbulkdata",
                    "location": "us",
                    "endpointInternal": "https://minio:9000",
                    "endpointExternal": "https://localhost:9000",
                    "auth" : {
                        "type": "hmac",
                        "accessKeyId": "minio",
                        "secretAccessKey": "change-password"
                    },
                    "disableBaseUrlValidation": true,
                    "exportPublic": true,
                    "disableOperationOutcomes": true,
                    "duplicationCheck": false, 
                    "validateResources": false, 
                    "create": false,
                    "presigned": true
                }
            }
        }

Create the /output/bulkdata folder

mkdir -p bulkdata

Download the ibm-fhir-server docker image

docker pull ibmcom/ibm-fhir-server:latest

$ docker pull ibmcom/ibm-fhir-server:latest
latest: Pulling from ibmcom/ibm-fhir-server
13897c84ca57: Pull complete 
64607cc74f9c: Pull complete 
16ac69f16a48: Pull complete 
54eaf4e89615: Pull complete 
215ce34bfb04: Pull complete 
23fc81994ef3: Pull complete 
358f256d50db: Pull complete 
2c5ee9f8d67e: Pull complete 
df1b462e3122: Pull complete 
64782415f212: Pull complete 
5871c358d552: Pull complete 
931b2daafdbb: Pull complete 
a77b0c08b2ad: Pull complete 
8405f4cee973: Pull complete 
a0d476c441e5: Pull complete 
d504580ab646: Pull complete 
b68755501c7d: Pull complete 
4071f846ff8d: Pull complete 
Digest: sha256:1829c8ab601355efecc190ef667fba93d92cbbd2e9305c109f736851c3ba97f4
Status: Downloaded newer image for ibmcom/ibm-fhir-server:latest
docker.io/ibmcom/ibm-fhir-server:latest

Start up the IBM FHIR Server with the fhir-server-config.json we just modified.

docker run --rm -d -p 9443:9443 -e BOOTSTRAP_DB=true \
    -v $(pwd)/fhir-server-config.json:/config/config/default/fhir-server-config.json \
    -v $(pwd)/bulkdata:/output/bulkdata \
    ibmcom/ibm-fhir-server

You’ll see the container id output 60a5f1cae6d677d80772f1736db1be74836a8a4845fcccc81286b7c557bc2d86.

Check that the applications are started using the container id.

$ docker logs 60a | grep -i started
[4/21/21, 20:55:58:449 UTC] 00000001 FrameworkMana I   CWWKE0002I: The kernel started after 1.43 seconds
[4/21/21, 20:55:58:464 UTC] 0000002a FeatureManage I   CWWKF0007I: Feature update started.
[4/21/21, 20:56:01:328 UTC] 00000030 AppMessageHel A   CWWKZ0001I: Application fhir-openapi started in 1.588 seconds.
[4/21/21, 20:56:03:141 UTC] 00000031 AppMessageHel A   CWWKZ0001I: Application fhir-bulkdata-webapp started in 3.402 seconds.
[4/21/21, 20:56:07:824 UTC] 0000002d AppMessageHel A   CWWKZ0001I: Application fhir-server-webapp started in 7.871 seconds.
[4/21/21, 20:56:07:868 UTC] 0000002a TCPPort       I   CWWKO0219I: TCP Channel defaultHttpEndpoint-ssl has been started and is now listening for requests on host *  (IPv4) port 9443.
[4/21/21, 20:56:07:880 UTC] 0000002a FeatureManage A   CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 10.885 seconds.

Download the Sample Data

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json -o Antonia30_Acosta403.json

Load the Sample Data bundle to the IBM FHIR Server

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--data-binary  "@Antonia30_Acosta403.json" -o response.json

Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.
Run the $import operation

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4/$import'     --header 'X-FHIR-TENANT-ID: default'     --header 'Content-Type: application/fhir+json'     --header 'X-FHIR-BULKDATA-PROVIDER: minio'     --header 'X-FHIR-BULKDATA-PROVIDER-OUTCOME: minio'     --user 'fhiruser:change-password'     --header 'Cookie: __cfduid=d02cd7c4e9e90b7f71c16bdd22da08fb41616718585'     --data-raw '{
        "resourceType": "Parameters",
        "id": "30321130-5032-49fb-be54-9b8b82b2445a",
        "parameter": [
            {
                "name": "inputSource",
                "valueUri": "https://my-server/source-fhir-server"
            },
            {
                "name": "inputFormat",
                "valueString": "application/fhir+ndjson"
            },
            {
                "name": "input",
                "part": [
                    {
                        "name": "type",
                        "valueString": "Patient"
                    },
                    {
                        "name": "url",
                        "valueUrl": "test-import.ndjson"
                    }
                ]
            },
            {
                "name": "storageDetail",
                "valueString": "ibm-cos"
            }
        ]
    }' -v

Grab the Location response header.

< HTTP/2 202
< content-location: https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=Zhkno6biH1Qwh2S6VDBT6w
< date: Fri, 23 Apr 2021 00:08:06 GMT
< content-length: 0
< content-language: en-US

Check the export status using the URL

curl -k --location --request GET 'https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=LqzauvqtHSmkpChVHo%2B1MQ' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --user 'fhiruser:change-password'

View the response and confirm successful export. You’ll see 4 successful imports and no failures.

{
    "transactionTime": "2021-04-23T00:22:51.368Z",
    "request": "https://localhost:9443/fhir-server/api/v4/$import",
    "requiresAccessToken": false,
    "output": [
        {
            "type": "OperationOutcome",
            "url": "test-import.ndjson_oo_success.ndjson",
            "count": 4
        }
    ],
    "error": [
        {
            "type": "OperationOutcome",
            "url": "test-import.ndjson_oo_errors.ndjson",
            "count": 0
        }
    ]
}

Check the Docker Logs docker logs 7050fdf7905e1fb320fd23b731639a21e064589c68d384e8add038879bc7993

[4/23/21, 0:19:30:986 UTC] 0000002e Reporter      I   Operation Type: $import
[4/23/21, 0:19:30:987 UTC] 0000002e Reporter      I   Resource Type         failures              success               processed             totalRead             totalValidation       totalWrite            fileSize              Resource Size
[4/23/21, 0:19:30:987 UTC] 0000002e Reporter      I   Patient               0                     4                     4                     2410                  0                     4795                  4303                  1075.0
[4/23/21, 0:19:30:988 UTC] 0000002e Reporter      I    ---- Total: 4 ImportRate: 0.49 ----

Run the $export operation

curl -k -v --location --request GET 'https://localhost:9443/fhir-server/api/v4/$export?_outputFormat=application/fhir+ndjson&_type=Patient' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --header 'X-FHIR-BULKDATA-PROVIDER: minio' \
    --header 'X-FHIR-BULKDATA-PROVIDER-OUTCOME: minio' \
    --user 'fhiruser:change-password'

Grab the Location response header, and check the export status

< HTTP/2 202
< content-location: https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=UKt4ESCnqOvAfxYWhdsfUg
< date: Fri, 23 Apr 2021 00:25:07 GMT
< content-length: 0
< content-language: en-US

Call the bulkdata-status endpoint

curl -k --location --request GET 'https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=UKt4ESCnqOvAfxYWhdsfUg' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --user 'fhiruser:change-password'

View the response, and download the exported file.

{
    "transactionTime": "2021-04-23T00:25:07.782Z",
    "request": "https://localhost:9443/fhir-server/api/v4/$export?_outputFormat=application/fhir+ndjson&_type=Patient",
    "requiresAccessToken": false,
    "output": [
        {
            "type": "Patient",
            "url": "https://localhost:9000/fhirbulkdata/hnFKv4BahpfK2iCnlPHOvnVknZVCVbgrBKwMn2HWCgg/Patient_1.ndjson?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=fhirAccessKey%2F20210423%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20210423T002551Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=21e52030687241b9380b8985d6ee4793328955a22bda3adabe0e724175947d58",
            "count": 4
        }
    ]
}

Download the file.

curl -k -o data.ndjson 'https://localhost:9000/fhirbulkdata/hnFKv4BahpfK2iCnlPHOvnVknZVCVbgrBKwMn2HWCgg/Patient_1.ndjson?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=fhirAccessKey%2F20210423%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20210423T002551Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=21e52030687241b9380b8985d6ee4793328955a22bda3adabe0e724175947d58'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4266  100  4266    0     0  20809      0 --:--:-- --:--:-- --:--:-- 20708

Look at the content and you’ll see it’s an ndjson.

You now have a working environment start-to-end of the Bulk Data on IBM FHIR Server.

Addendum

File Provider

By changing X-FHIR-BULKDATA-PROVIDER to default, you can use the mapped volume /output/bulkdata to export and import from a local directory.

Logs

You can see the job logs using:

docker logs nervous_boyd

docker exec -it nervous_boyd cat /logs/joblogs/<path to job>

Networking issues

If you hit issues where the two images don’t talk to each other, please connect them to the same network:

docker network connect –alias fhir docker_default nervous_boyd

docker network connect –alias minio docker_default minio

2021-04-22

Question: How to setup the IBM FHIR Server with fhir-audit?

The IBM FHIR Server supports audit events for FHIR operations (CREATE-READ-UPDATE-DELETE-OPERATION) in Cloud Auditing Data Federation (CADF) and HL7 FHIR AuditEvent and pushing the events to an Apache Kafka backend. You can read more about it on the IBM FHIR Server site.

Let’s spin up an IBM FHIR Server with fhir-audit and see what we get with a running container.

Create an Event Streams instance on IBM Cloud Console
Create a topic FHIR_AUDIT

Click Service Credentials, click Create New Credential, Click Add

Expand your new Service Credential, and copy the JSON and store locally, we’ll use it in a subsequent step.

Download the fhir-server-config.json

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/fhir-server-config-audit-environment.json -o fhir-server-config-audit-environment.json

The file contains a setting for loading the properties from the Environment, and some basic settings:

        "audit": {
            "serviceClassName" : "com.ibm.fhir.audit.impl.KafkaService",
            "serviceProperties" : {
                "auditTopic": "FHIR_AUDIT",
                "geoCity": "Dallas",
                "geoState": "TX",
                "geoCounty": "US"
            }
        }

Download the docker image

docker pull ibmcom/ibm-fhir-server:latest

$ docker pull ibmcom/ibm-fhir-server:latest
latest: Pulling from ibmcom/ibm-fhir-server
13897c84ca57: Pull complete 
64607cc74f9c: Pull complete 
16ac69f16a48: Pull complete 
54eaf4e89615: Pull complete 
215ce34bfb04: Pull complete 
23fc81994ef3: Pull complete 
358f256d50db: Pull complete 
2c5ee9f8d67e: Pull complete 
df1b462e3122: Pull complete 
64782415f212: Pull complete 
5871c358d552: Pull complete 
931b2daafdbb: Pull complete 
a77b0c08b2ad: Pull complete 
8405f4cee973: Pull complete 
a0d476c441e5: Pull complete 
d504580ab646: Pull complete 
b68755501c7d: Pull complete 
4071f846ff8d: Pull complete 
Digest: sha256:1829c8ab601355efecc190ef667fba93d92cbbd2e9305c109f736851c3ba97f4
Status: Downloaded newer image for ibmcom/ibm-fhir-server:latest
docker.io/ibmcom/ibm-fhir-server:latest

Make the JSON a single line

cat << EOF | tr -d '\n'
{
  "api_key": "credentialapikey",
  "apikey": "credentialapikey",
  "iam_apikey_description": "Auto-generated for key 01ba3165-85d9-410b-ad1a-1111",
  "iam_apikey_name": "Service credentials-1",
  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Manager",
  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/11111::serviceid:ServiceId-864b549f-ff90-4b24-84f2-1111",
  "instance_id": "6b00cc8c-26ec-4c42-a1e8-f4da5b9f71e7",
  "kafka_admin_url": "https://admin.eventstreams.cloud.ibm.com",
  "kafka_brokers_sasl": [
    "broker-3.eventstreams.cloud.ibm.com:9093",
    "broker-5.eventstreams.cloud.ibm.com:9093",
    "broker-1.eventstreams.cloud.ibm.com:9093",
    "broker-2.eventstreams.cloud.ibm.com:9093",
    "broker-4.eventstreams.cloud.ibm.com:9093",
    "broker-0.eventstreams.cloud.ibm.com:9093"
  ],
  "kafka_http_url": "https://admin.eventstreams.cloud.ibm.com",
  "password": "credentialapikey",
  "user": "token"
}
EOF

The output is:

{  "api_key": "credentialapikey",  "apikey": "credentialapikey",  "iam_apikey_description": "Auto-generated for key 01ba3165-85d9-410b-ad1a-1111",  "iam_apikey_name": "Service credentials-1",  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Manager",  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/11111::serviceid:ServiceId-864b549f-ff90-4b24-84f2-1111",  "instance_id": "6b00cc8c-26ec-4c42-a1e8-f4da5b9f71e7",  "kafka_admin_url": "https://admin.eventstreams.cloud.ibm.com",  "kafka_brokers_sasl": [    "broker-3.eventstreams.cloud.ibm.com:9093",    "broker-5.eventstreams.cloud.ibm.com:9093",    "broker-1.eventstreams.cloud.ibm.com:9093",    "broker-2.eventstreams.cloud.ibm.com:9093",    "broker-4.eventstreams.cloud.ibm.com:9093",    "broker-0.eventstreams.cloud.ibm.com:9093"  ],  "kafka_http_url": "https://admin.eventstreams.cloud.ibm.com",  "password": "credentialapikey",  "user": "token"}

Export ES_CONFIG

export ES_CONFIG='{  "api_key": "credentialapikey",  "apikey": "credentialapikey",  "iam_apikey_description": "Auto-generated for key 01ba3165-85d9-410b-ad1a-1111",  "iam_apikey_name": "Service credentials-1",  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Manager",  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/11111::serviceid:ServiceId-864b549f-ff90-4b24-84f2-1111",  "instance_id": "6b00cc8c-26ec-4c42-a1e8-f4da5b9f71e7",  "kafka_admin_url": "https://admin.eventstreams.cloud.ibm.com",  "kafka_brokers_sasl": [    "broker-3.eventstreams.cloud.ibm.com:9093",    "broker-5.eventstreams.cloud.ibm.com:9093",    "broker-1.eventstreams.cloud.ibm.com:9093",    "broker-2.eventstreams.cloud.ibm.com:9093",    "broker-4.eventstreams.cloud.ibm.com:9093",    "broker-0.eventstreams.cloud.ibm.com:9093"  ],  "kafka_http_url": "https://admin.eventstreams.cloud.ibm.com",  "password": "credentialapikey",  "user": "token"}'

Note the single quotes surround the above.

Start up the IBM FHIR Server with the EventStreams credentials and fhir-server-config.json we just downloaded.

docker run --rm -d -p 9443:9443 -e BOOTSTRAP_DB=true \
    -v $(pwd)/fhir-server-config-audit-environment.json:/config/config/default/fhir-server-config.json \
    -e EVENT_STREAMS_AUDIT_BINDING="${ES_CONFIG}" ibmcom/ibm-fhir-server

You’ll see the container id output 60a5f1cae6d677d80772f1736db1be74836a8a4845fcccc81286b7c557bc2d86.

Check that the applications are started using the container id.

$ docker logs 60a | grep -i started
[4/21/21, 20:55:58:449 UTC] 00000001 FrameworkMana I   CWWKE0002I: The kernel started after 1.43 seconds
[4/21/21, 20:55:58:464 UTC] 0000002a FeatureManage I   CWWKF0007I: Feature update started.
[4/21/21, 20:56:01:328 UTC] 00000030 AppMessageHel A   CWWKZ0001I: Application fhir-openapi started in 1.588 seconds.
[4/21/21, 20:56:03:141 UTC] 00000031 AppMessageHel A   CWWKZ0001I: Application fhir-bulkdata-webapp started in 3.402 seconds.
[4/21/21, 20:56:07:824 UTC] 0000002d AppMessageHel A   CWWKZ0001I: Application fhir-server-webapp started in 7.871 seconds.
[4/21/21, 20:56:07:868 UTC] 0000002a TCPPort       I   CWWKO0219I: TCP Channel defaultHttpEndpoint-ssl has been started and is now listening for requests on host *  (IPv4) port 9443.
[4/21/21, 20:56:07:880 UTC] 0000002a FeatureManage A   CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 10.885 seconds.

Download the Sample Data

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json -o Antonia30_Acosta403.json

Load the Sample Data bundle to the IBM FHIR Server

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--header "Authorization: Basic ${DUMMY_PASSWORD}" \
--data-binary  "@Antonia30_Acosta403.json" -o response.json

Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.

Note: DUMMY_PASSWORD should be set to your environment.

Download Kafka from Apache Kafka
Create a client-ssl.properties from your service credentials.

cat << EOF > client-ssl.properties
bootstrap.servers=broker-2.mybroker.com:9093,broker-5.mybroker.com:9093,broker-0.mybroker.com:9093,broker-3.mybroker.com:9093,broker-4.mybroker.com:9093,broker-1.mybroker.com:9093
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="MYPASSWORD";
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
ssl.protocol=TLSv1.2
EOF

Unzip the kafka archive
Check the Kafka Console Consumer

bash bin/kafka-console-consumer.sh \
    --bootstrap-server broker-4.mybroker.com:9093,broker-5.mybroker.com:9093,broker-3.mybroker.com:9093,broker-2.mybroker.com:9093,broker-1.mybroker.com:9093,broker-0.mybroker.com:9093 \
    --topic FHIR_AUDIT --max-messages 25 --property print.timestamp=true --consumer.config client-ssl.properties --from-beginning

We see up to 25 messages related to the generated data.

You see the CADF audit messages, and you can change settings in the fhir-server-config.json to use AuditEvent. You can see an example at https://github.com/IBM/FHIR/blob/main/fhir-server/liberty-config/config/default/fhir-server-config-audit-config.json

2021-04-21

Question: How to use fhir-path-cli?

In IBM FHIR Server 4.7.0, the IBM FHIR Server team introduced fhir-path-cli and related to a blog on $everything.

Now, let’s download some sample data on the server using the Integration Test data.

Download the Sample Data

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json -o Antonia30_Acosta403.json

Load the Sample Data bundle to the IBM FHIR Server

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--header "Authorization: Basic ${DUMMY_PASSWORD}" \
--data-binary  "@Antonia30_Acosta403.json" -o response.json

Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.
Check the response.json and find the id for Patient (it should be the first one).

        {
            "response": {
                "id": "178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003",
                "status": "201",
                "location": "Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/_history/1",
                "etag": "W/\"1\"",
                "lastModified": "2021-04-21T13:34:50.207684Z"
            }
        },

Check the Patient/[id]/$everything with a _type constraint and _since.

Request *

curl -k --location --request GET 'https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/$everything?_type=CarePlan,CareTeam&_since=2021-01-01T00:00:00Z&_count=1' \
    --header 'Content-Type: application/fhir+json' \
    --header "Authorization: Basic ${DUMMY_PASSWORD}" -o care.json

Response *

{
    "resourceType": "Bundle",
    "id": "689856f9-6ef6-4d74-8f77-a813ff7b1c6d",
    "type": "searchset",
    "total": 11,
    "entry": [
        {
            "fullUrl": "https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003",
            "resource": {
                "resourceType": "Patient",
                "id": "178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003",
                "meta": {
                    "versionId": "1",
                    "lastUpdated": "2021-04-21T13:34:50.207684Z"
                }
                ...
            }
        }
    ]
}

You’ll see further down CareTeam and CarePlan resources…

This is where we can have a bit of fun… download the fhir-path-cli

curl -L https://github.com/IBM/FHIR/releases/download/4.7.0/fhir-path-4.7.0-cli.jar -o fhir-path-4.7.0-cli.jar

Let’s run the fhir-path-4.7.0-cli.jar and test a FHIRPath.

Command Line

java -jar fhir-path-4.7.0-cli.jar --path 'entry.fullUrl' --file care.json

Output

https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CarePlan/178f4a3f86a-1a9a1d73-4d1d-46d5-8494-339dddbfe9d2
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CarePlan/178f4a3f86b-06c07a6c-3be7-4e6a-abb2-040ff67c8c1f
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CarePlan/178f4a3f873-d8069835-938e-45b1-9a70-5aba3794f6f3
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CarePlan/178f4a3f87c-ce204596-8037-416c-b2e0-fc7442d9a276
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CarePlan/178f4a3f87c-46785bc7-807a-40e0-bc08-792977ac8ec8
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CareTeam/178f4a3f86a-b79251c3-5019-426e-8211-748adbdfdf54
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CareTeam/178f4a3f86b-22a7baab-77b3-4fb5-b318-9249ea80ec85
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CareTeam/178f4a3f873-e370fba2-f45e-4cdd-b13b-781e249f84f8
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CareTeam/178f4a3f87c-596d80cc-7b42-4054-bc02-a48acfcc95b7
https://localhost:9443/fhir-server/api/v4/Patient/178f4a3f869-6abfe1e6-3a4a-4c9a-81a8-fb8c6263e003/CareTeam/178f4a3f87c-6cc197cd-b665-46e0-ad35-c016edeaf13b

You’ve seen a brief introduction to fhir-operation-everything and a bonus fhir-path-cli showing the output.

Note, DUMMY_PASSWORD should be replaced with or set as a value when the corresponding curl steps are executed.

Thanks to my colleague that implemented $everything.

2021-04-21

Question: How to use IBM FHIR Server Notifications with IBM Cloud Functions?

A question I had recently was “How to use IBM FHIR Server” with IBM Cloud Functions….

Here is the recipe:

1 – Create a topic FHIR_NOTIFICATION (for instance in EventStreams)

2 – Setup a IBM Cloud Function per https://github.com/IBM/ibm-cloud-functions-message-hub-trigger

3 – Update your process-message with the following code:

function main(params) {
  return new Promise((resolve, reject) => {
      console.log(params.messages);
    if (!params.messages || !params.messages[0] || !params.messages[0].value) {
      reject("Invalid arguments. Must include 'messages' JSON array with 'value' field");
    }
    const msgs = params.messages;
    const locations = [];
    for (let i = 0; i < msgs.length; i++) {
      const msg = msgs[i];
      console.log(msg.value.location);
      locations.push(msg.value.location);
    }
    resolve({
      locations,
    });
  });
}

exports.main = main;

exports.main = main;

4 – Invoke with parameters

    {
      "messages": [
        {
          "key": null,
          "offset": 0,
          "partition": 2,
          "topic": "FHIR_NOTIFICATION",
          "value": {
            "datasourceId": "default",
            "lastUpdated": "2021-04-16T13:55:13.312415Z",
            "location": "SearchParameter/178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac/_history/1",
            "operationType": "create",
            "resource": {
              "base": [
                "Encounter"
              ],
              "code": "diff-start-end-time",
              "description": "Example of sophisiticated extraction",
              "expression": "iif(period.start.exists() and period.end.exists(), between(period.start , period.end, 'days'), {})",
              "id": "178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac",
              "meta": {
                "lastUpdated": "2021-04-16T13:55:13.312415Z",
                "versionId": "1"
              },
              "name": "Example of sophisiticated extraction",
              "resourceType": "SearchParameter",
              "status": "draft",
              "type": "quantity",
              "url": "https://fhir.ibm.com/demo/diff-start-end-time"
            },
            "resourceId": "178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac",
            "tenantId": "default"
          }
        }
      ]
    }

4 – Click Invoke

5 – Look to the Right to see the logging/activation (Expand the tick on the right-middle of the screen)

6 – You’ll see you now have a Function connected with EventStreams.

Notes

The activation dashboard is important https://cloud.ibm.com/functions/dashboard. You can see great detail on the logs and what is passed to the Function.

{
  "activationId": "960f129da9f04f138f129da9f0ef13f7",
  "annotations": [
    {
      "key": "path",
      "value": "14d753c7-8620-4caa-b6af-3934e33ae451/fhir-event-streams/process-message"
    },
    {
      "key": "waitTime",
      "value": 274
    },
    {
      "key": "kind",
      "value": "nodejs:12"
    },
    {
      "key": "timeout",
      "value": false
    },
    {
      "key": "limits",
      "value": {
        "concurrency": 1,
        "logs": 10,
        "memory": 256,
        "timeout": 60000
      }
    },
    {
      "key": "initTime",
      "value": 25
    }
  ],
  "duration": 32,
  "end": 1618583048738,
  "logs": [
    "2021-04-16T14:24:08.737951Z    stdout: [",
    "2021-04-16T14:24:08.737997Z    stdout: {",
    "2021-04-16T14:24:08.738031Z    stdout: key: null,",
    "2021-04-16T14:24:08.738037Z    stdout: offset: 0,",
    "2021-04-16T14:24:08.738042Z    stdout: partition: 2,",
    "2021-04-16T14:24:08.738047Z    stdout: topic: 'FHIR_NOTIFICATION',",
    "2021-04-16T14:24:08.738052Z    stdout: value: {",
    "2021-04-16T14:24:08.738057Z    stdout: datasourceId: 'default',",
    "2021-04-16T14:24:08.738062Z    stdout: lastUpdated: '2021-04-16T13:55:13.312415Z',",
    "2021-04-16T14:24:08.738066Z    stdout: location: 'SearchParameter/178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac/_history/1',",
    "2021-04-16T14:24:08.738071Z    stdout: operationType: 'create',",
    "2021-04-16T14:24:08.738076Z    stdout: resource: [Object],",
    "2021-04-16T14:24:08.738081Z    stdout: resourceId: '178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac',",
    "2021-04-16T14:24:08.738085Z    stdout: tenantId: 'default'",
    "2021-04-16T14:24:08.738134Z    stdout: }",
    "2021-04-16T14:24:08.738141Z    stdout: }",
    "2021-04-16T14:24:08.738146Z    stdout: ]",
    "2021-04-16T14:24:08.738152Z    stdout: SearchParameter/178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac/_history/1"
  ],
  "name": "process-message",
  "namespace": "14d753c7-8620-4caa-b6af-3934e33ae451",
  "publish": false,
  "response": {
    "result": {
      "msgs": [
        {
          "key": null,
          "offset": 0,
          "partition": 2,
          "topic": "FHIR_NOTIFICATION",
          "value": {
            "datasourceId": "default",
            "lastUpdated": "2021-04-16T13:55:13.312415Z",
            "location": "SearchParameter/178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac/_history/1",
            "operationType": "create",
            "resource": {
              "base": [
                "Encounter"
              ],
              "code": "diff-start-end-time",
              "description": "Example of sophisiticated extraction",
              "expression": "iif(period.start.exists() and period.end.exists(), between(period.start , period.end, 'days'), {})",
              "id": "178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac",
              "meta": {
                "lastUpdated": "2021-04-16T13:55:13.312415Z",
                "versionId": "1"
              },
              "name": "Example of sophisiticated extraction",
              "resourceType": "SearchParameter",
              "status": "draft",
              "type": "quantity",
              "url": "https://fhir.ibm.com/demo/diff-start-end-time"
            },
            "resourceId": "178daf6d720-d2da7624-1664-4e50-a4f8-e558ab667fac",
            "tenantId": "default"
          }
        }
      ]
    },
    "size": 877,
    "status": "success",
    "success": true
  },
  "start": 1618583048706,
  "subject": "pb",
  "version": "0.0.1"
}

2021-04-16

Question: How do I decode the Job Id from the Bulk Data Export or Import on the IBM FHIR Server?

Question: How do I decode the Job Id from the Bulk Data Export or Import on the IBM FHIR Server?

You can use the following code to walk through and decode your job id using your passowrd.

2021-04-15

Question: How do I configure fhir-notification Kafka so I don’t have to use a keystore?

The IBM FHIR Server provides an eventing service that notifies about persistence events – CUD (Create-Update-Delete). The notification service can trigger specific actions in a downstream application. You can configure these events to flow to Apache Kafka.

If you want to configure the IBM FHIR Server without a keystore, you can configure SASL_SSL like the below (notice there is no keystore specified).

{
    "fhirServer":{
        "notifications":{
            "kafka": {
                "enabled": true,
                "topicName": "FHIR_NOTIFICATIONS",
                "connectionProperties": {
                    "bootstrap.servers": "broker-1.mybroker.com:9093,broker-2.mybroker.com:9093,broker-0.mybroker.com:9093,broker-5.mybroker.com:9093,broker-4.mybroker.com:9093,broker-3.mybroker.com:9093",
                    "security.protocol": "SASL_SSL",
                    "sasl.mechanism": "PLAIN",
                    "ssl.protocol": "TLSv1.2",
                    "ssl.enabled.protocols": "TLSv1.2",
                    "ssl.endpoint.identification.algorithm": "HTTPS",
                    "security.inter.broker.protocol": "SSL",
                    "sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"token\" password=\"MYPASSWORD\";",
                    "acks": "all",
                    "retries": "60",
                    "request.timeout.ms": "10000",
                    "max.block.ms": "60000",
                    "max.in.flight.requests.per.connection": "5",
                    "client.dns.lookup": "use_all_dns_ips"
                }
            }
        }
    }
}

If you run a workload against IBM FHIR Server, it’ll generate a set of notifications…

Download Kafka from https://kafka.apache.org
Create a client-ssl.properties

cat << EOF > client-ssl.properties
bootstrap.servers=broker-2.mybroker.com:9093,broker-5.mybroker.com:9093,broker-0.mybroker.com:9093,broker-3.mybroker.com:9093,broker-4.mybroker.com:9093,broker-1.mybroker.com:9093
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="MYPASSWORD";
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
ssl.protocol=TLSv1.2
EOF

Unzip the kafka archive
Check the Kafka Console Consumer

bash bin/kafka-console-consumer.sh --bootstrap-server broker-4.mybroker.com:9093,broker-5.mybroker.com:9093,broker-3.mybroker.com:9093,broker-2.mybroker.com:9093,broker-1.mybroker.com:9093,broker-0.mybroker.com:9093 --topic FHIR_NOTIFICATION --max-messages 25 --property print.timestamp=true --consumer.config client-ssl.properties

2021-04-15

Category: IBM FHIR Server

Checking a Postgres Function Definition

Tracing Bulk Data with Cloud Object Storage

rclone sync

Extract the references from the first subject

Extract the references from the first subject alternative

Select the first Patient reference where the Diastolic is over 90

Select the Patient’s first name where the reference matches the logical id

Setup Minio

Setup IBM FHIR Server

Addendum