Tag: bulkdata

  • Recipe: Setting up IBM FHIR Server and Azure in Development

    The IBM FHIR Server has support for exporting and importing Bulk Data using extended operations for Bulk Data $import, $export and $bulkdata-status, which are implemented as Java Maven projects. The IBM FHIR Server uses JSR252 JavaBatch jobs running in the Open Liberty Java Batch Framework to enable access to Large Volumes of HL7 FHIR data.

    This blog is a follow on to Recipe: IBM FHIR Server – Using Bulk Data with the Azure Blob Service, and provides a docker-compose file that works with the Azure emulator called Azurite.

    Typically, you can run the container locally:

    docker run -p 10000:10000 -v /local/path/to/azurite:/data mcr.microsoft.com/azure-storage/azurite \
        azurite-blob --blobHost 0.0.0.0 --blobPort 10000

    Recipe

    1. Pull the image

    $docker pull mcr.microsoft.com/azure-storage/azurite
    
    Using default tag: latest
    latest: Pulling from azure-storage/azurite
    396c31837116: Pull complete 
    9e7b0c9574dd: Pull complete 
    ec07c04a8d4c: Pull complete 
    c1eb01e62785: Pull complete 
    2cbc599970e9: Pull complete 
    a0ee56369073: Pull complete 
    ad1956587082: Pull complete 
    29652032eab7: Pull complete 
    Digest: sha256:4d40e97bf9345c9e321f4b8cf722dc4615d5d6080fd2953844be288a13eadb59
    Status: Downloaded newer image for mcr.microsoft.com/azure-storage/azurite:latest
    mcr.microsoft.com/azure-storage/azurite:latest

    2. Download the docker-compose.yml and put it in a working folder

    3. Download the fhir-server-config.json and put it in the same working folder

    4. Create a folder azurite in the working folder.

    5. The file layout should look like the following:

    6. Startup the Docker Compose

    nerdctl --address /var/run/docker/containerd/containerd.sock compose up

    You can then upload data, and use the Azurite emulator, the key is:

    "storageProviders": {
         "default" : {
         "type": "azure-blob",
         "bucketName": "fhirbulkdata",
         "auth" : {
              "type": "connection",
              "connection": "DefaultEndpointsProtocol=http;AccountName=account1;AccountKey=key1;BlobEndpoint=http://azure-blob:10000/account1;"
         },
         "disableOperationOutcomes": true,
         "duplicationCheck": false, 
         "validateResources": false, 
         "create": false
         }
    }
  • Bulk Data: Using the SMART-on-FHIR Bulk Data Client to test $export

    I recently attended the HL7 FHIR Connectathon 29. For those that are not familiar with Connectathons, I think they are fairly unique events featuring standards enthusiasts, vendors and implementors doing hands-on standards development (FHIR) and testing. As an attendee I picked one of the tracksbulk data.

    This blog is part of a series on Bulk Data Setup and Testing based on my experience at HL7 FHIR Connectathon 29.

    The SMART-on-FHIR team has built a node application to call Bulk Data $export and facilitate the calls to get a bearer token.

    1. Clone the Bulk Data Client repository
    /git/wffh/2023$ git clone -b main-pb --single-branch \
        https://github.com/prb112/bulk-data-client.git && \
        cd $(basename $_ .git)
    Cloning into 'bulk-data-client'...
    remote: Enumerating objects: 343, done.
    remote: Counting objects: 100% (343/343), done.
    remote: Compressing objects: 100% (188/188), done.
    remote: Total 343 (delta 220), reused 261 (delta 146), pack-reused 0
    Receiving objects: 100% (343/343), 407.93 KiB | 3.32 MiB/s, done.
    Resolving deltas: 100% (220/220), done.
    /git/wffh/2023/bulk-data-client$
    

    Note I added a changes to support the IBM FHIR Server.

    1. Install nvm
    $ brew install nvm
    Running `brew update --preinstall`...
    ==> Auto-updated Homebrew!
    Updated 3 taps (homebrew/core, homebrew/cask and minio/stable).
    ==> New Formulae
    cwb3                                                      erofs-utils                                               netmask                                                   scalingo                                                  zk
    ==> Updated Formulae
    Updated 390 formulae.
    
    ...
    
    nvm 0.35.3 is already installed but outdated (so it will be upgraded).
    ==> Downloading https://ghcr.io/v2/homebrew/core/nvm/manifests/0.39.1_1
    ######################################################################## 100.0%
    ==> Downloading https://ghcr.io/v2/homebrew/core/nvm/blobs/sha256:6e14c8a2bf94212545c1ebac9a722df168c318d0e8af2fc75b729a07fea54efe
    ==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:6e14c8a2bf94212545c1ebac9a722df168c318d0e8af2fc75b729a07fea54efe?se=2022-01-13T21%3A30%3A00Z&sig=52CYOAyuxabqLNYih9%2BjMQtHLFrN%2FVyHXf%2FgPuBgi6w%3D&sp=r&spr=https&sr=b&sv=2019-12-12
    ######################################################################## 100.0%
    ==> Upgrading nvm
      0.35.3 -> 0.39.1_1
    
    ==> Pouring nvm--0.39.1_1.all.bottle.tar.gz
    ...
    
    Bash completion has been installed to:
      /usr/local/etc/bash_completion.d
    ==> Summary
    🍺  /usr/local/Cellar/nvm/0.39.1_1: 9 files, 184.1KB
    ==> Running `brew cleanup nvm`...
    Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
    Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
    Removing: /usr/local/Cellar/nvm/0.35.3... (7 files, 149.9KB)
    
    1. Switch to Node 16
    $ nvm use 16
    Now using node v16.13.0 (npm v8.1.2)
    
    1. Install Node 16
    $ nvm install
    Found '/git/wffh/2023/bulk-data-client/.nvmrc' with version <16>
    Downloading and installing node v16.13.2...
    Downloading https://nodejs.org/dist/v16.13.2/node-v16.13.2-darwin-x64.tar.xz...
    ############################################################## 100.0%
    Computing checksum with sha256sum
    Checksums matched!
    Now using node v16.13.2 (npm v8.1.2)
    
    1. Create the binaries
    $ npm run clean && npm run build
    
    > bulk-data-client@1.0.0 clean
    > rm -rf ./built
    
    
    > bulk-data-client@1.0.0 build
    > tsc
    
    1. Create the config/ibm-fhir-server.js – update the fhirUrl, tokenUrl and privateKey. I used the RS384 Key.
    1. Run the application
    $ AUTO_RETRY_TRANSIENT_ERRORS=1 SHOW_ERRORS=1 node . --config config/ibm-fhir-server.js  --_type=Patient
    Kick-off started
    Got new access token
    Kick-off completed
    Bulk Data export started
    Bulk Data export completed in 10 seconds
    Export manifest:  {
      transactionTime: '2022-01-14T20:56:31.553Z',
      request: 'https://bulk.cluster1-blue-x123456.containers.appdomain.cloud/fhir-server/api/v4/Patient/$export?_type=Patient',
      requiresAccessToken: false,
      output: [
        {
          type: 'Patient',
          url: 'https://x123456.cloud-object-storage.appdomain.cloud/x123456/f1wSMD84-3eRVNTUlyIwctWtNxA33Odz14KiQctry-U/Patient_1.ndjson?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=x123456%2F20220114%2Fus-east%2Fs3%2Faws4_request&X-Amz-Date=20220114T205641Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=abcdefgh',
          count: 2
        }
      ],
      error: []
    }
    
    Downloading exported files: β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰ 100%
              Downloaded Files: 1 of 1
                FHIR Resources: 2
                   Attachments: 0
               Downloaded Size: 6.3 kB
    
    Download completed in 0 seconds
    Do you want to signal the server that this export can be removed? [Y/n]n
    

    If you need to debug, you’ll see all the outputs:

    NODE_DEBUG=* AUTO_RETRY_TRANSIENT_ERRORS=1 SHOW_ERRORS=1 node . \
       --config config/ibm-fhir-server.js
    
    1. The files are downloaded to a local directory, and you can check the data. cat downloads/1.Patient.ndjson
    {"resourceType":"Patient","id":"17e44643c84-7618d4fe-e1d3-48e5-8a66-923bd4780c17","meta":{"versionId":"1","lastUpdated":"2022-01-10T14:28:46.854836Z"},"text":{"status":"generated","div":"<div xmlns=\"http://www.w3.org/1999/xhtml\">Generated by <a href=\"https://github.com/synthetichealth/synthea\">Synthea</a>.Version identifier: master-branch-latest-2-g8e7e92c\n .   Person seed: 7105486291024734541  Population seed: 0</div>"},"extension":[{"url":"http://hl7.org/fhir/StructureDefinition/patient-mothersMaidenName","valueString":"Graciela518 Badillo851"},{"url":"http://hl7.org/fhir/StructureDefinition/patient-birthPlace","valueAddress":{"city":"Caguas","state":"Puerto Rico","country":"PR"}},{"url":"http://synthetichealth.github.io/synthea/disability-adjusted-life-years","valueDecimal":1.6165934938317164},{"url":"http://synthetichealth.github.io/synthea/quality-adjusted-life-years","valueDecimal":47.383406506168285},{"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex","valueCode":"F"},{"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex","valueCode":"M"}],"identifier":[{"system":"https://github.com/synthetichealth/synthea","value":"bd958c64-56b6-f206-b03d-8f4b8a417215"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"MR","display":"Medical Record Number"}],"text":"Medical Record Number"},"system":"http://hospital.smarthealthit.org","value":"bd958c64-56b6-f206-b03d-8f4b8a417215"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"SS","display":"Social Security Number"}],"text":"Social Security Number"},"system":"http://hl7.org/fhir/sid/us-ssn","value":"999-93-6090"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"DL","display":"Driver's License"}],"text":"Driver's License"},"system":"urn:oid:2.16.840.1.113883.4.3.25","value":"S99913499"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"PPN","display":"Passport Number"}],"text":"Passport Number"},"system":"http://standardhealthrecord.org/fhir/StructureDefinition/passportNumber","value":"X26583229X"}],"name":[{"use":"official","family":"Acosta403","given":["Antonia30"],"prefix":["Mrs."]},{"use":"maiden","family":"Armenta418","given":["Antonia30"],"prefix":["Mrs."]}],"telecom":[{"system":"phone","value":"555-457-3489","use":"home"}],"gender":"female","birthDate":"1970-09-06","address":[{"extension":[{"extension":[{"url":"latitude","valueDecimal":41.6497330022825},{"url":"longitude","valueDecimal":-71.17511064024423}],"url":"http://hl7.org/fhir/StructureDefinition/geolocation"}],"line":["635 Littel Esplanade Suite 65"],"city":"Fall River","state":"Massachusetts","postalCode":"02790","country":"US"}],"maritalStatus":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v3-MaritalStatus","code":"M","display":"M"}],"text":"M"},"multipleBirthBoolean":false,"communication":[{"language":{"coding":[{"system":"urn:ietf:bcp:47","code":"es","display":"Spanish"}],"text":"Spanish"}}]}
    {"resourceType":"Patient","id":"17e4f2f3dca-67724e2c-3991-4ac1-b6f9-51fc30209894","meta":{"versionId":"1","lastUpdated":"2022-01-12T16:46:43.149786Z"},"text":{"status":"generated","div":"<div xmlns=\"http://www.w3.org/1999/xhtml\">Generated by <a href=\"https://github.com/synthetichealth/synthea\">Synthea</a>.Version identifier: v2.4.0-404-ge7ce2295\n .   Person seed: -8292973307042192125  Population seed: 0</div>"},"extension":[{"extension":[{"url":"ombCategory","valueCoding":{"system":"urn:oid:2.16.840.1.113883.6.238","code":"2135-2","display":"Other"}},{"url":"text","valueString":"Other"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"},{"extension":[{"url":"ombCategory","valueCoding":{"system":"urn:oid:2.16.840.1.113883.6.238","code":"2186-5","display":"Not Hispanic or Latino"}},{"url":"text","valueString":"Not Hispanic or Latino"}],"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"},{"url":"http://hl7.org/fhir/StructureDefinition/patient-mothersMaidenName","valueString":"Isabel214 ResΓ©ndez908"},{"url":"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex","valueCode":"M"},{"url":"http://hl7.org/fhir/StructureDefinition/patient-birthPlace","valueAddress":{"city":"La Paz","state":"Baja California","country":"MX"}},{"url":"http://synthetichealth.github.io/synthea/disability-adjusted-life-years","valueDecimal":0.13213216767824654},{"url":"http://synthetichealth.github.io/synthea/quality-adjusted-life-years","valueDecimal":34.86786783232176}],"identifier":[{"system":"https://github.com/synthetichealth/synthea","value":"d171d808-1f31-4ad3-aba5-e03a2fa921c7"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"MR","display":"Medical Record Number"}],"text":"Medical Record Number"},"system":"http://hospital.smarthealthit.org","value":"d171d808-1f31-4ad3-aba5-e03a2fa921c7"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"SS","display":"Social Security Number"}],"text":"Social Security Number"},"system":"http://hl7.org/fhir/sid/us-ssn","value":"999-83-6585"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"DL","display":"Driver's License"}],"text":"Driver's License"},"system":"urn:oid:2.16.840.1.113883.4.3.25","value":"S99914532"},{"type":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v2-0203","code":"PPN","display":"Passport Number"}],"text":"Passport Number"},"system":"http://standardhealthrecord.org/fhir/StructureDefinition/passportNumber","value":"X18544570X"}],"name":[{"use":"official","family":"Hurtado459","given":["Marco Antonio298"],"prefix":["Mr."]}],"telecom":[{"system":"phone","value":"555-185-5373","use":"home"}],"gender":"male","birthDate":"1983-04-03","address":[{"extension":[{"extension":[{"url":"latitude","valueDecimal":42.49384661818001},{"url":"longitude","valueDecimal":-70.92858466579901}],"url":"http://hl7.org/fhir/StructureDefinition/geolocation"}],"line":["552 Rippin Port"],"city":"Revere","state":"Massachusetts","postalCode":"02151","country":"US"}],"maritalStatus":{"coding":[{"system":"http://terminology.hl7.org/CodeSystem/v3-MaritalStatus","code":"M","display":"M"}],"text":"M"},"multipleBirthBoolean":false,"communication":[{"language":{"coding":[{"system":"urn:ietf:bcp:47","code":"es","display":"Spanish"}],"text":"Spanish"}}]}
    

    As you can see the bulk-data-client is a handy tool to test Bulk Data with $export with SMART Backend Services Authorization.

  • Bulk Data Configurations for IBM FHIR Server’s Storage Providers

    As of IBM FHIR Server 4.10.2

    A colleague of mine is entering into the depths of the IBM FHIR Server’s Bulk Data feature. Each tenant in the IBM FHIR Server may specify multiple storageProviders. The default tenant is assumed, unless specified with the Http Headers X-FHIR-BULKDATA-PROVIDER and X-FHIR-BULKDATA-PROVIDER-OUTCOME. Each tenant’s configuration may mix the different providers, however each provider is only of a single type. For instance, minio is aws-s3 and default is file and az is azure-blob.

    Note, type http is only applicable to $import operations. Export is only supported with s3, azure-blob and file.

    File Storage Provider Configuration

    The file storage provider uses a directory local to the IBM FHIR Server. The fileBase is an absolute path that must exist. Each import inputUrl is a relative path from the fileBase. The File Provider is available for both import and export. Authentication is not supported.

    {
        "__comment": "IBM FHIR Server BulkData - File Storage configuration",
        "fhirServer": {
            "bulkdata": {
                "__comment" : "The other bulkdata configuration elements are skipped",
                "storageProviders": {
                    "default": {
                        "type": "file",
                        "fileBase": "/opt/wlp/usr/servers/fhir-server/output",
                        "disableOperationOutcomes": true,
                        "duplicationCheck": false,
                        "validateResources": false,
                        "create": false
                    }
                }
            }
        }
    }
    

    An example request is:

    {
        "resourceType": "Parameters",
        "id": "30321130-5032-49fb-be54-9b8b82b2445a",
        "parameter": [
            {
                "name": "inputSource",
                "valueUri": "https://my-server/source-fhir-server"
            },
            {
                "name": "inputFormat",
                "valueString": "application/fhir+ndjson"
            },
            {
                "name": "input",
                "part": [
                    {
                        "name": "type",
                        "valueString": "AllergyIntolerance"
                    },
                    {
                        "name": "url",
                        "valueUrl": "r4_AllergyIntolerance.ndjson"
                    }
                ]
            },
            {
                "name": "storageDetail",
                "valueString": "file"
            }
        ]
    }
    

    HTTPS Storage Provider Configuration

    The http storage provider uses a set of validBaseUrls to confirm $import inputUrl is acceptable. The Https Provider is available for import. Authentication is not supported.

    {
        "__comment": "IBM FHIR Server BulkData - Https Storage configuration (Import only)",
        "fhirServer": {
            "bulkdata": {
                "__comment" : "The other bulkdata configuration elements are skipped",
                "storageProviders": {
                    "default": {
                        "type": "https",
                        "__comment": "The whitelist of valid base urls, you can always disable",
                        "validBaseUrls": [],
                        "disableBaseUrlValidation": true,
                        "__comment": "You can always direct to another provider",
                        "disableOperationOutcomes": true,
                        "duplicationCheck": false,
                        "validateResources": false
                    }
                }
            }
        }
    }
    

    An example request is:

    {
        "resourceType": "Parameters",
        "id": "30321130-5032-49fb-be54-9b8b82b2445a",
        "parameter": [
            {
                "name": "inputSource",
                "valueUri": "https://my-server/source-fhir-server"
            },
            {
                "name": "inputFormat",
                "valueString": "application/fhir+ndjson"
            },
            {
                "name": "input",
                "part": [
                    {
                        "name": "type",
                        "valueString": "AllergyIntolerance"
                    },
                    {
                        "name": "url",
                        "valueUrl": "https://validbaseurl.com/r4_AllergyIntolerance.ndjson"
                    }
                ]
            },
            {
                "name": "storageDetail",
                "valueString": "https"
            }
        ]
    }
    

    Azure Storage Provider Configuration

    The azure-blob storage provider uses a connection string from the Azure Blob configuration. The bucketName is the blob storage name. The azure-blob provider supports import and export. Authentication and configuration are built into the Connection string.

        "__comment": "IBM FHIR Server BulkData - Azure Blob Storage configuration",
        "fhirServer": {
            "bulkdata": {
                "__comment" : "The other bulkdata configuration elements are skipped",
                "storageProviders": {
                    "default": {
                        "type": "azure-blob",
                        "bucketName": "fhirtest",
                        "auth": {
                            "type": "connection",
                            "connection": "DefaultEndpointsProtocol=https;AccountName=fhirdt;AccountKey=ABCDEF==;EndpointSuffix=core.windows.net"
                        },
                        "disableBaseUrlValidation": true,
                        "disableOperationOutcomes": true
                    }
                }
            }
        }
    }```

    An example request is:

    {
        "resourceType": "Parameters",
        "id": "30321130-5032-49fb-be54-9b8b82b2445a",
        "parameter": [
            {
                "name": "inputSource",
                "valueUri": "https://my-server/source-fhir-server"
            },
            {
                "name": "inputFormat",
                "valueString": "application/fhir+ndjson"
            },
            {
                "name": "input",
                "part": [
                    {
                        "name": "type",
                        "valueString": "AllergyIntolerance"
                    },
                    {
                        "name": "url",
                        "valueUrl": "r4_AllergyIntolerance.ndjson"
                    }
                ]
            },
            {
                "name": "storageDetail",
                "valueString": "azure-blob"
            }
        ]
    }
    

    S3 Storage Provider Configuration

    The aws-s3 storage provider supports import and export. The bucketName, location, auth style (hmac, iam), endpointInternal, endpointExternal are separate values in the configuration. Note, enableParquet is obsolete.

    {
        "__comment": "IBM FHIR Server BulkData - AWS/COS/Minio S3 Storage configuration",
        "fhirServer": {
            "bulkdata": {
                "__comment" : "The other bulkdata configuration elements are skipped",
                "storageProviders": {
                    "default": {
                        "type": "aws-s3",
                        "bucketName": "myfhirbucket",
                        "location": "us-east-2",
                        "endpointInternal": "https://s3.us-east-2.amazonaws.com",
                        "endpointExternal": "https://myfhirbucket.s3.us-east-2.amazonaws.com",
                        "auth": {
                            "type": "hmac",
                            "accessKeyId": "AKIAAAAF2TOAAATMAAAO",
                            "secretAccessKey": "mmmUVsqKzAAAAM0QDSxH9IiaGQAAA"
                        },
                        "enableParquet": false,
                        "disableBaseUrlValidation": true,
                        "exportPublic": false,
                        "disableOperationOutcomes": true,
                        "duplicationCheck": false,
                        "validateResources": false,
                        "create": false,
                        "presigned": true,
                        "accessType": "host"
                    }
                }
            }
        }
    }
    

    An example request is:

    {
        "resourceType": "Parameters",
        "id": "30321130-5032-49fb-be54-9b8b82b2445a",
        "parameter": [
            {
                "name": "inputSource",
                "valueUri": "https://my-server/source-fhir-server"
            },
            {
                "name": "inputFormat",
                "valueString": "application/fhir+ndjson"
            },
            {
                "name": "input",
                "part": [
                    {
                        "name": "type",
                        "valueString": "AllergyIntolerance"
                    },
                    {
                        "name": "url",
                        "valueUrl": "r4_AllergyIntolerance.ndjson"
                    }
                ]
            },
            {
                "name": "storageDetail",
                "valueString": "aws-s3"
            }
        ]
    }

    Note, you can exchange aws-s3 and ibm-cos as the parameter.where(name=’storageDetail’). These are treated interchangeably.

    There are lots of configurations that are possible, I hope this helps you.

  • IBM FHIR Server – Debugging Tips

    Checking a Postgres Function Definition

    I needed to verify my postgres function.

    1. Shows the contents of the functions in the schema.
    SELECT pg_get_functiondef(f.oid)
    FROM pg_catalog.pg_proc f
    INNER JOIN pg_catalog.pg_namespace n ON (f.pronamespace = n.oid)
    WHERE n.nspname = 'fhirdata';
    
    1. Show all the details of the functions in the schema.
    SELECT *
    FROM pg_catalog.pg_proc f
    INNER JOIN pg_catalog.pg_namespace n ON (f.pronamespace = n.oid)
    WHERE n.nspname = 'fhirdata';
    

    Tracing Bulk Data with Cloud Object Storage

    I wanted to figure out why my code was failing to connect to the backend S3 bucket.

    I used the environment variable in my docker image called TRACE_SPEC. TRACE_SPEC is loaded into the logging as the traceSpecification.

    I set this to *=info:com.ibm.cloud.*=FINEST which spits out great detail to S3 using the IBM COS SDK.

    The output looks like:

    With this level of trace, you can really dive into the connection, and determine what is going on.

    Note, if you want the whole picture of what is happening with COS and JavaBatch and BulkData, use the following:

    *=info:com.ibm.fhir.*=finer:RRA=all:WAS.j2c=all:com.ibm.ws.jdbc.*=all:Transaction=all:ConnLeakLogic=all:Transaction=all:com.ibm.ws.transaction.services.WebAppTransactionCollaboratorImpl=all:RRA=all:com.ibm.cloud.*=FINEST

  • BulkData: rclone with S3

    This was super helpful for debugging S3 files/folders.

    brew install rclone
    rclone rcd --config rclone.conf  --rc-web-gui --no-check-certificate --rc-user rc --rc-pass rc
    
    [minio]
    type = s3
    env_auth = false
    access_key_id = minio
    secret_access_key = demooo-password
    region = us-east-1
    endpoint = https://localhost:9000
    location_constraint = 
    server_side_encryption = 
    

    rclone sync

    rclone sync --config rclone-s3.conf --no-check-certificate -i fhir-performance:fhir-performance $(pwd)