Extract the references from the first subject

Bundle.entry.resource.ofType(Observation).subject[0].reference.value

Output Patient/ebb16c62-cb06-4ff8-8ce8-ccb865e7a240

Extract the references from the first subject alternative

Bundle.entry.resource.ofType(Observation).subject.first().reference.value

Output Patient/ebb16c62-cb06-4ff8-8ce8-ccb865e7a240

Select the first Patient reference where the Diastolic is over 90

Bundle.entry.resource.ofType(Observation).where(component.where(code.coding.code = '8462-4').value.where( value > 90.0).exists()).subject.reference.value.first()

Output Patient/5fabfccd-254e-42af-bed2-84199a5c05f2

Select the Patient’s first name where the reference matches the logical id

Bundle.entry.resource.ofType(Patient).where(id = '" + v.split("/")[1] + "').name.given.value

Output John

In IBM FHIR Server 4.6.0, IBM FHIR Server has refactored the supported Bulk Data operations that support the HL7 FHIR BulkDataAccess IG: STU1 and the Proposal for $import Operation.

To demonstrate the new features in a quick script, use the following to setup an S3 demonstration from the start-to-end.

• Setup Minio
• Setup IBM FHIR Server
• Load Sample Data
• Run $import
• Run $export

Setup Minio

1. Pull the minio docker image

2. Create a local data directory for data

mkdir -p minio-data/

3. Download demonstration only certificates/keys

curl -L -o private.key raw.githubusercontent.com/IBM/FHIR/main/build/docker/minio/private.key

curl -L -o public.crt https://raw.githubusercontent.com/IBM/FHIR/main/build/docker/minio/public.crt

Note, these are demonstration only Private Key and Public cert pair.

3. Create the Minio container

export MINIO_ROOT_PASSWORD=$(openssl rand -base64 20)
docker run -d --rm -p 9000:9000 \
  --name minio \
  -h minio \
  -e "MINIO_ROOT_USER=fhir-s3-admin" \
  -e "MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}" \
  -e "MINIO_ACCESS_KEY=fhirAccessKey" \
  -e "MINIO_SECRET_KEY=${MINIO_ROOT_PASSWORD}" \
  -v $(pwd)/minio-data:/data \
  -v $(pwd)/public.crt:/root/.minio/certs/public.crt \
  -v $(pwd)/private.key:/root/.minio/certs/private.key \
  minio/minio server /data

Note, the Minio container will create local certificates.

4. Using your MINIO_ROOT_PASSWORD echo ${MINIO_ROOT_PASSWORD}, log in to your Minio instance.

5. Create a Bucket (folder), look in the lower left. Name it fhirbulkdata.

Setup IBM FHIR Server

8. Download the fhir-server-config.json

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/fhir-server-config.json -o fhir-server-config.json

9. Update the fhir-server-config.json fhirServer/bulkdata with the following snippet to use hmac and bulkdata. Be sure to update fhirServer/bulkdata/storageProviders/minio/accessKeyId to fhirAccessKey and fhirServer/bulkdata/storageProviders/minio/secretAccessKey to your MINIO_ROOT_PASSWORD.

    "bulkdata": {
            "enabled": true,
            "core": {
                "api": {
                    "url": "https://localhost:9443/ibm/api/batch",
                    "user": "fhiradmin",
                    "password": "change-password",
                    "truststore": "resources/security/fhirTrustStore.p12",
                    "truststorePassword": "change-password", 
                    "trustAll": true
                },
                "cos" : { 
                    "partUploadTriggerSizeMB": 10,
                    "objectSizeThresholdMB": 200,
                    "objectResourceCountThreshold": 200000,
                    "useServerTruststore": true
                },
                "file" : { 
                    "writeTriggerSizeMB": 1,
                    "sizeThresholdMB": 200,
                    "resourceCountThreshold": 200000
                },
                "pageSize": 100,
                "batchIdEncryptionKey": "change-password",
                "maxPartitions": 3, 
                "maxInputs": 5
            },
            "storageProviders": {
                "default" : {
                    "type": "file",
                    "_type": "ibm-cos|aws-s3|file|https",
                    "validBaseUrls": [],
                    "fileBase": "/output/bulkdata",
                    "disableBaseUrlValidation": true,
                    "exportPublic": true,
                    "disableOperationOutcomes": true,
                    "duplicationCheck": false, 
                    "validateResources": false
                },
                "minio" : {
                    "type": "aws-s3",
                    "bucketName": "fhirbulkdata",
                    "location": "us",
                    "endpointInternal": "https://minio:9000",
                    "endpointExternal": "https://localhost:9000",
                    "auth" : {
                        "type": "hmac",
                        "accessKeyId": "minio",
                        "secretAccessKey": "change-password"
                    },
                    "disableBaseUrlValidation": true,
                    "exportPublic": true,
                    "disableOperationOutcomes": true,
                    "duplicationCheck": false, 
                    "validateResources": false, 
                    "create": false,
                    "presigned": true
                }
            }
        }

10. Create the /output/bulkdata folder

mkdir -p bulkdata

11. Download the ibm-fhir-server docker image

docker pull ibmcom/ibm-fhir-server:latest

$ docker pull ibmcom/ibm-fhir-server:latest
latest: Pulling from ibmcom/ibm-fhir-server
13897c84ca57: Pull complete 
64607cc74f9c: Pull complete 
16ac69f16a48: Pull complete 
54eaf4e89615: Pull complete 
215ce34bfb04: Pull complete 
23fc81994ef3: Pull complete 
358f256d50db: Pull complete 
2c5ee9f8d67e: Pull complete 
df1b462e3122: Pull complete 
64782415f212: Pull complete 
5871c358d552: Pull complete 
931b2daafdbb: Pull complete 
a77b0c08b2ad: Pull complete 
8405f4cee973: Pull complete 
a0d476c441e5: Pull complete 
d504580ab646: Pull complete 
b68755501c7d: Pull complete 
4071f846ff8d: Pull complete 
Digest: sha256:1829c8ab601355efecc190ef667fba93d92cbbd2e9305c109f736851c3ba97f4
Status: Downloaded newer image for ibmcom/ibm-fhir-server:latest
docker.io/ibmcom/ibm-fhir-server:latest

12. Start up the IBM FHIR Server with the fhir-server-config.json we just modified.

docker run --rm -d -p 9443:9443 -e BOOTSTRAP_DB=true \
    -v $(pwd)/fhir-server-config.json:/config/config/default/fhir-server-config.json \
    -v $(pwd)/bulkdata:/output/bulkdata \
    ibmcom/ibm-fhir-server

You’ll see the container id output 60a5f1cae6d677d80772f1736db1be74836a8a4845fcccc81286b7c557bc2d86.

13. Check that the applications are started using the container id.

$ docker logs 60a | grep -i started
[4/21/21, 20:55:58:449 UTC] 00000001 FrameworkMana I   CWWKE0002I: The kernel started after 1.43 seconds
[4/21/21, 20:55:58:464 UTC] 0000002a FeatureManage I   CWWKF0007I: Feature update started.
[4/21/21, 20:56:01:328 UTC] 00000030 AppMessageHel A   CWWKZ0001I: Application fhir-openapi started in 1.588 seconds.
[4/21/21, 20:56:03:141 UTC] 00000031 AppMessageHel A   CWWKZ0001I: Application fhir-bulkdata-webapp started in 3.402 seconds.
[4/21/21, 20:56:07:824 UTC] 0000002d AppMessageHel A   CWWKZ0001I: Application fhir-server-webapp started in 7.871 seconds.
[4/21/21, 20:56:07:868 UTC] 0000002a TCPPort       I   CWWKO0219I: TCP Channel defaultHttpEndpoint-ssl has been started and is now listening for requests on host *  (IPv4) port 9443.
[4/21/21, 20:56:07:880 UTC] 0000002a FeatureManage A   CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 10.885 seconds.

14. Download the Sample Data

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json -o Antonia30_Acosta403.json

15. Load the Sample Data bundle to the IBM FHIR Server

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--data-binary  "@Antonia30_Acosta403.json" -o response.json

16. Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.

17. Run the $import operation

19. Check the export status using the URL

curl -k --location --request GET 'https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=LqzauvqtHSmkpChVHo%2B1MQ' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --user 'fhiruser:change-password'

19. View the response and confirm successful export. You’ll see 4 successful imports and no failures.

{
    "transactionTime": "2021-04-23T00:22:51.368Z",
    "request": "https://localhost:9443/fhir-server/api/v4/$import",
    "requiresAccessToken": false,
    "output": [
        {
            "type": "OperationOutcome",
            "url": "test-import.ndjson_oo_success.ndjson",
            "count": 4
        }
    ],
    "error": [
        {
            "type": "OperationOutcome",
            "url": "test-import.ndjson_oo_errors.ndjson",
            "count": 0
        }
    ]
}

20. Check the Docker Logs docker logs 7050fdf7905e1fb320fd23b731639a21e064589c68d384e8add038879bc7993

[4/23/21, 0:19:30:986 UTC] 0000002e Reporter      I   Operation Type: $import
[4/23/21, 0:19:30:987 UTC] 0000002e Reporter      I   Resource Type         failures              success               processed             totalRead             totalValidation       totalWrite            fileSize              Resource Size
[4/23/21, 0:19:30:987 UTC] 0000002e Reporter      I   Patient               0                     4                     4                     2410                  0                     4795                  4303                  1075.0
[4/23/21, 0:19:30:988 UTC] 0000002e Reporter      I    ---- Total: 4 ImportRate: 0.49 ----

20. Run the $export operation

curl -k -v --location --request GET 'https://localhost:9443/fhir-server/api/v4/$export?_outputFormat=application/fhir+ndjson&_type=Patient' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --header 'X-FHIR-BULKDATA-PROVIDER: minio' \
    --header 'X-FHIR-BULKDATA-PROVIDER-OUTCOME: minio' \
    --user 'fhiruser:change-password'

21. Grab the Location response header, and check the export status

< HTTP/2 202
< content-location: https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=UKt4ESCnqOvAfxYWhdsfUg
< date: Fri, 23 Apr 2021 00:25:07 GMT
< content-length: 0
< content-language: en-US

22. Call the bulkdata-status endpoint

curl -k --location --request GET 'https://localhost:9443/fhir-server/api/v4/$bulkdata-status?job=UKt4ESCnqOvAfxYWhdsfUg' \
    --header 'X-FHIR-TENANT-ID: default' \
    --header 'Content-Type: application/fhir+json' \
    --user 'fhiruser:change-password'

23. View the response, and download the exported file.

{
    "transactionTime": "2021-04-23T00:25:07.782Z",
    "request": "https://localhost:9443/fhir-server/api/v4/$export?_outputFormat=application/fhir+ndjson&_type=Patient",
    "requiresAccessToken": false,
    "output": [
        {
            "type": "Patient",
            "url": "https://localhost:9000/fhirbulkdata/hnFKv4BahpfK2iCnlPHOvnVknZVCVbgrBKwMn2HWCgg/Patient_1.ndjson?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=fhirAccessKey%2F20210423%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20210423T002551Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=21e52030687241b9380b8985d6ee4793328955a22bda3adabe0e724175947d58",
            "count": 4
        }
    ]
}

24. Download the file.

curl -k -o data.ndjson 'https://localhost:9000/fhirbulkdata/hnFKv4BahpfK2iCnlPHOvnVknZVCVbgrBKwMn2HWCgg/Patient_1.ndjson?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=fhirAccessKey%2F20210423%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20210423T002551Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=21e52030687241b9380b8985d6ee4793328955a22bda3adabe0e724175947d58'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4266  100  4266    0     0  20809      0 --:--:-- --:--:-- --:--:-- 20708

25. Look at the content and you’ll see it’s an ndjson.