The IBM FHIR Server enables user defined and implementation guide defined SearchParameter definitions and is constantly improving Specification conformance. Given these dynamic changes, the IBM FHIR Server Search parameter values may require refresh to optimize the Search and Retrieval of the data in the operational data store.

A quick tool to aid in this is fhir-bucket

Let me show you how to run reindexing with the IBM FHIR Server container Docker: ibmcom/ibm-fhir-server. This feature requires 4.9.0 or higher.

Recipe

1. Prior to 4.9.0, build the Maven Projects and the Docker Build. You should see [INFO] BUILD SUCCESS after each Maven build, and docker.io/ibmcom/ibm-fhir-server:latest when the Docker build is successful.

mvn clean install -f fhir-examples -B -DskipTests -ntp
mvn clean install -f fhir-parent -B -DskipTests -ntp
docker build -t ibmcom/ibm-fhir-server:latest fhir-install

2. Download the fhir-server-config.json

curl -L -o fhir-server-config.json \
    https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/fhir-server-config.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  8423  100  8423    0     0  40495      0 --:--:-- --:--:-- --:--:-- 40301

3. Download the extension-search-parameters.json

curl -L -o extension-search-parameters.json \
    https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/extension-search-parameters.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   154  100   154    0     0    611      0 --:--:-- --:--:-- --:--:--   611

4. Start the Docker container, and capture the container id. It’s going to take a few moments to start up as it lays down the test database.

docker run -d -p 9443:9443 -e BOOTSTRAP_DB=true \
  -v $(pwd)/fhir-server-config.json:/config/config/default/fhir-server-config.json \
  -v $(pwd)/extension-search-parameters.json:/config/config/default/extension-search-parameters.json \
  ibmcom/ibm-fhir-server
9c44ad49784a6ca6456799ab63aea460fc564cc4a9bd5f4fef1aeae4b82225bd

4. Check the logs until you see:

docker logs 9c44ad49784a6ca6456799ab63aea460fc564cc4a9bd5f4fef1aeae4b82225bd
...
[6/16/21, 15:31:34:533 UTC] 0000002a FeatureManage A   CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 17.665 seconds.

5. Download the Sample Data

curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json \
-o Antonia30_Acosta403.json

6. Load the Sample Data bundle to the IBM FHIR Server

curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--data-binary  "@Antonia30_Acosta403.json" -o response.json

Note, DUMMY_PASSWORD should be previously set.

7. Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.

cat response.json | jq -r '.entry[].response.status' | sort -u
201

8. Search maiden-name = Graciela518 Badillo851, and you’ll see the reindexing completes and the search succeeds.

curl --location --request GET 'https://localhost:9443/fhir-server/api/v4/Patient?maiden-name=Graciela518 Badillo851' \
--header 'X-FHIR-TENANT-ID: default' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--header 'Content-Type: application/json' -k

{ 
    "resourceType":"OperationOutcome",
    "id":"ac-11-0-3-14c966ad-8c8b-438d-86ad-de4b2fdb27b0",
    "issue":[
        {
            "severity":"fatal",
            "code":"invalid",
            "details": {
                "text":"Search parameter 'maiden-name' for resource type 'Patient' was not found."
            },
            "expression":["<empty>"]
        }
    ]
}

9. Download the extension-search-parameters.json with an example expression.

curl -L -o extension-search-parameters.json \
    https://gist.githubusercontent.com/prb112/03f0d77f72685180868d2f31a1070ebe/raw/14fda1cb86dda74095faa5e32fec6f9226ffb587/extension-search-parameters.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   752  100   752    0     0   2231      0 --:--:-- --:--:-- --:--:--  2231

10. Post 4.9.0’s release, download the fhir-bucket-cli

curl -L -o fhir-bucket-4.9.0-cli.jar \
    https://repo1.maven.org/maven2/com/ibm/fhir/fhir-bucket/4.9.0/fhir-bucket-4.9.0-cli.jar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 46.7M  100 46.7M    0     0  12.2M      0  0:00:03  0:00:03 --:--:-- 12.2M

11. Pre 4.9.0’s release, copy the fhir-bucket-cli from the .m2 repository.

cp ~/.m2/repository/com/ibm/fhir/fhir-bucket/4.9.0-SNAPSHOT/fhir-bucket-4.9.0-SNAPSHOT-cli.jar fhir-bucket-4.9.0-cli.jar

12. Download the client truststore

curl -L -o fhirClientTrustStore.p12 \
    https://github.com/IBM/FHIR/raw/main/fhir-server-test/src/test/resources/fhirClientTrustStore.p12
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   174  100   174    0     0    661      0 --:--:-- --:--:-- --:--:--   661
100  5946  100  5946    0     0  12651      0 --:--:-- --:--:-- --:--:-- 12651

13. Download the fhir-bucket logging.properties

curl -L -o logging.properties \
    https://raw.githubusercontent.com/IBM/FHIR/main/fhir-bucket/logging.properties
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   688  100   688    0     0   3245      0 --:--:-- --:--:-- --:--:--  3245

14. Create the bucket.properties file

cat << EOF > bucket.properties
fhir.server.host=localhost
fhir.server.port=9443
fhir.server.user=fhiruser
fhir.server.pass=change-password
fhir.server.endpoint=/fhir-server/api/v4/
truststore=fhirClientTrustStore.p12
truststore.pass=change-password
read.timeout=125000
connect.timeout=20000
pool.connections.max=400
disable.hostname.verification=true
EOF

15. Execute the fhir-bucket reindex operation with the current date-time (e.g. now is 2021-12-01T00:00:00Z)

export JAR="fhir-bucket-4.9.0-cli.jar"
java -Djava.util.logging.config.file=logging.properties -jar "${JAR}" \
    --fhir-properties bucket.properties \
    --tenant-name default \
    --max-concurrent-fhir-requests 100 \
    --no-scan \
    --reindex-tstamp 2021-12-01T00:00:00Z \
    --reindex-resource-count 50 \
    --reindex-concurrent-requests 20 \
    --reindex-client-side-driven

16. Wait until the reindexing threads complete.

2021-08-05 15:28:05.629 00000001    INFO x.ClientDrivenReindexOperation Starting monitor thread
2021-08-05 15:28:07.864 0000001b    INFO x.ClientDrivenReindexOperation called $retrieve-index: 200 OK [took 0.851 s]
2021-08-05 15:28:07.872 0000001b    INFO x.ClientDrivenReindexOperation Index IDs available for processing - filling worker pool
2021-08-05 15:28:09.145 0000001d    INFO x.ClientDrivenReindexOperation called $reindex (indexIds=2,...): 200 OK [took 1.268 s]
...
2021-08-05 15:28:27.946 0000001b    INFO x.ClientDrivenReindexOperation called $retrieve-index (afterIndexId=1802): 200 OK [took 0.053 s]
2021-08-05 15:28:27.949 0000001b    INFO x.ClientDrivenReindexOperation No more index IDs to retrieve
2021-08-05 15:28:27.950 0000001b    INFO x.ClientDrivenReindexOperation Nothing left to do, so tell all the worker threads to exit
2021-08-05 15:28:27.951 0000001b    INFO x.ClientDrivenReindexOperation Waiting for 20 threads to complete before exiting
2021-08-05 15:28:32.952 0000001b    INFO x.ClientDrivenReindexOperation Reindexing was completed
2021-08-05 15:29:32.891 0000001a    INFO   com.ibm.fhir.bucket.app.Main Stopping all services

17. Search maiden-name = Graciela518 Badillo851, and you’ll see the reindexing completes and the search succeeds.

curl --location --request GET 'https://localhost:9443/fhir-server/api/v4/Patient?maiden-name=Graciela518 Badillo851' \
--header 'X-FHIR-TENANT-ID: default' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--header 'Content-Type: application/json' -k

    {
            "id": "17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
            "fullUrl": "https://localhost:9443/fhir-server/api/v4/Patient/17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
            "resource": {
                "resourceType": "Patient",
                "id": "17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
                "meta": {
                    "versionId": "1",
                    "lastUpdated": "2021-08-05T19:16:37.89885Z"
                },
                "text": {
                    "status": "generated",
                    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\">Generated by <a href=\"https://github.com/synthetichealth/synthea\">Synthea</a>.Version identifier: master-branch-latest-2-g8e7e92c\n .   Person seed: 7105486291024734541  Population seed: 0</div>"
                },
                "extension": [
                    {
                        "url": "http://hl7.org/fhir/StructureDefinition/patient-mothersMaidenName",
                        "valueString": "Graciela518 Badillo851"
                    },
                    {
                        "url": "http://hl7.org/fhir/StructureDefinition/patient-birthPlace",
                        "valueAddress": {
                            "city": "Caguas",
                            "state": "Puerto Rico",
                            "country": "PR"
                        }
                    },
                    {
                        "url": "http://synthetichealth.github.io/synthea/disability-adjusted-life-years",
                        "valueDecimal": 1.6165934938317164
                    },
                    {
                        "url": "http://synthetichealth.github.io/synthea/quality-adjusted-life-years",
                        "valueDecimal": 47.383406506168288
                    },
                    {
                        "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex",
                        "valueCode": "F"
                    },
                    {
                        "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex",
                        "valueCode": "M"
                    }
                ],
                "identifier": [
                    {
                        "system": "https://github.com/synthetichealth/synthea",
                        "value": "bd958c64-56b6-f206-b03d-8f4b8a417215"
                    },
                    {
                        "type": {
                            "coding": [
                                {
                                    "system": "http://terminology.hl7.org/CodeSystem/v2-0203",
                                    "code": "MR",
                                    "display": "Medical Record Number"
                                }
                            ],
                            "text": "Medical Record Number"
                        },
                        "system": "http://hospital.smarthealthit.org",
                        "value": "bd958c64-56b6-f206-b03d-8f4b8a417215"
                    },
                    {
                        "type": {
                            "coding": [
                                {
                                    "system": "http://terminology.hl7.org/CodeSystem/v2-0203",
                                    "code": "SS",
                                    "display": "Social Security Number"
                                }
                            ],
                            "text": "Social Security Number"
                        },
                        "system": "http://hl7.org/fhir/sid/us-ssn",
                        "value": "999-93-6090"
                    },
            }
        ]

References

• IBM FHIR Server: User’s Guide