The IBM FHIR Server enables user defined and implementation guide defined SearchParameter definitions and is constantly improving Specification conformance. Given these dynamic changes, the IBM FHIR Server Search parameter values may require refresh to optimize the Search and Retrieval of the data in the operational data store.
A quick tool to aid in this is fhir-bucket
Let me show you how to run reindexing with the IBM FHIR Server container Docker: ibmcom/ibm-fhir-server. This feature requires 4.9.0 or higher.
Recipe
- Prior to 4.9.0, build the Maven Projects and the Docker Build. You should see
[INFO] BUILD SUCCESS after each Maven build, and docker.io/ibmcom/ibm-fhir-server:latest when the Docker build is successful.
mvn clean install -f fhir-examples -B -DskipTests -ntp
mvn clean install -f fhir-parent -B -DskipTests -ntp
docker build -t ibmcom/ibm-fhir-server:latest fhir-install
- Download the fhir-server-config.json
curl -L -o fhir-server-config.json \
https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/fhir-server-config.json
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8423 100 8423 0 0 40495 0 --:--:-- --:--:-- --:--:-- 40301
- Download the extension-search-parameters.json
curl -L -o extension-search-parameters.json \
https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server/liberty-config/config/default/extension-search-parameters.json
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 154 100 154 0 0 611 0 --:--:-- --:--:-- --:--:-- 611
- Start the Docker container, and capture the container id. It’s going to take a few moments to start up as it lays down the test database.
docker run -d -p 9443:9443 -e BOOTSTRAP_DB=true \
-v $(pwd)/fhir-server-config.json:/config/config/default/fhir-server-config.json \
-v $(pwd)/extension-search-parameters.json:/config/config/default/extension-search-parameters.json \
ibmcom/ibm-fhir-server
9c44ad49784a6ca6456799ab63aea460fc564cc4a9bd5f4fef1aeae4b82225bd
- Check the logs until you see:
docker logs 9c44ad49784a6ca6456799ab63aea460fc564cc4a9bd5f4fef1aeae4b82225bd
...
[6/16/21, 15:31:34:533 UTC] 0000002a FeatureManage A CWWKF0011I: The defaultServer server is ready to run a smarter planet. The defaultServer server started in 17.665 seconds.
- Download the Sample Data
curl -L https://raw.githubusercontent.com/IBM/FHIR/main/fhir-server-test/src/test/resources/testdata/everything-operation/Antonia30_Acosta403.json \
-o Antonia30_Acosta403.json
- Load the Sample Data bundle to the IBM FHIR Server
curl -k --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Content-Type: application/fhir+json' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--data-binary "@Antonia30_Acosta403.json" -o response.json
Note, DUMMY_PASSWORD should be previously set.
- Scan the response.json for any status that is not "status": "201". For example, the status is in the family of User Request Error or Server Side Error.
cat response.json | jq -r '.entry[].response.status' | sort -u
201
- Search maiden-name = Graciela518 Badillo851, and you’ll see the reindexing completes and the search succeeds.
curl --location --request GET 'https://localhost:9443/fhir-server/api/v4/Patient?maiden-name=Graciela518 Badillo851' \
--header 'X-FHIR-TENANT-ID: default' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--header 'Content-Type: application/json' -k
{
"resourceType":"OperationOutcome",
"id":"ac-11-0-3-14c966ad-8c8b-438d-86ad-de4b2fdb27b0",
"issue":[
{
"severity":"fatal",
"code":"invalid",
"details": {
"text":"Search parameter 'maiden-name' for resource type 'Patient' was not found."
},
"expression":["<empty>"]
}
]
}
- Download the extension-search-parameters.json with an example expression.
curl -L -o extension-search-parameters.json \
https://gist.githubusercontent.com/prb112/03f0d77f72685180868d2f31a1070ebe/raw/14fda1cb86dda74095faa5e32fec6f9226ffb587/extension-search-parameters.json
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 752 100 752 0 0 2231 0 --:--:-- --:--:-- --:--:-- 2231
- Post 4.9.0’s release, download the fhir-bucket-cli
curl -L -o fhir-bucket-4.9.0-cli.jar \
https://repo1.maven.org/maven2/com/ibm/fhir/fhir-bucket/4.9.0/fhir-bucket-4.9.0-cli.jar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 46.7M 100 46.7M 0 0 12.2M 0 0:00:03 0:00:03 --:--:-- 12.2M
- Pre 4.9.0’s release, copy the
fhir-bucket-cli from the .m2 repository.
cp ~/.m2/repository/com/ibm/fhir/fhir-bucket/4.9.0-SNAPSHOT/fhir-bucket-4.9.0-SNAPSHOT-cli.jar fhir-bucket-4.9.0-cli.jar
- Download the client truststore
curl -L -o fhirClientTrustStore.p12 \
https://github.com/IBM/FHIR/raw/main/fhir-server-test/src/test/resources/fhirClientTrustStore.p12
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 174 100 174 0 0 661 0 --:--:-- --:--:-- --:--:-- 661
100 5946 100 5946 0 0 12651 0 --:--:-- --:--:-- --:--:-- 12651
- Download the fhir-bucket logging.properties
curl -L -o logging.properties \
https://raw.githubusercontent.com/IBM/FHIR/main/fhir-bucket/logging.properties
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 688 100 688 0 0 3245 0 --:--:-- --:--:-- --:--:-- 3245
- Create the bucket.properties file
cat << EOF > bucket.properties
fhir.server.host=localhost
fhir.server.port=9443
fhir.server.user=fhiruser
fhir.server.pass=change-password
fhir.server.endpoint=/fhir-server/api/v4/
truststore=fhirClientTrustStore.p12
truststore.pass=change-password
read.timeout=125000
connect.timeout=20000
pool.connections.max=400
disable.hostname.verification=true
EOF
- Execute the fhir-bucket reindex operation with the current date-time (e.g. now is
2021-12-01T00:00:00Z)
export JAR="fhir-bucket-4.9.0-cli.jar"
java -Djava.util.logging.config.file=logging.properties -jar "${JAR}" \
--fhir-properties bucket.properties \
--tenant-name default \
--max-concurrent-fhir-requests 100 \
--no-scan \
--reindex-tstamp 2021-12-01T00:00:00Z \
--reindex-resource-count 50 \
--reindex-concurrent-requests 20 \
--reindex-client-side-driven
- Wait until the reindexing threads complete.
2021-08-05 15:28:05.629 00000001 INFO x.ClientDrivenReindexOperation Starting monitor thread
2021-08-05 15:28:07.864 0000001b INFO x.ClientDrivenReindexOperation called $retrieve-index: 200 OK [took 0.851 s]
2021-08-05 15:28:07.872 0000001b INFO x.ClientDrivenReindexOperation Index IDs available for processing - filling worker pool
2021-08-05 15:28:09.145 0000001d INFO x.ClientDrivenReindexOperation called $reindex (indexIds=2,...): 200 OK [took 1.268 s]
...
2021-08-05 15:28:27.946 0000001b INFO x.ClientDrivenReindexOperation called $retrieve-index (afterIndexId=1802): 200 OK [took 0.053 s]
2021-08-05 15:28:27.949 0000001b INFO x.ClientDrivenReindexOperation No more index IDs to retrieve
2021-08-05 15:28:27.950 0000001b INFO x.ClientDrivenReindexOperation Nothing left to do, so tell all the worker threads to exit
2021-08-05 15:28:27.951 0000001b INFO x.ClientDrivenReindexOperation Waiting for 20 threads to complete before exiting
2021-08-05 15:28:32.952 0000001b INFO x.ClientDrivenReindexOperation Reindexing was completed
2021-08-05 15:29:32.891 0000001a INFO com.ibm.fhir.bucket.app.Main Stopping all services
- Search maiden-name = Graciela518 Badillo851, and you’ll see the reindexing completes and the search succeeds.
curl --location --request GET 'https://localhost:9443/fhir-server/api/v4/Patient?maiden-name=Graciela518 Badillo851' \
--header 'X-FHIR-TENANT-ID: default' \
--user "fhiruser:${DUMMY_PASSWORD}" \
--header 'Content-Type: application/json' -k
{
"id": "17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
"fullUrl": "https://localhost:9443/fhir-server/api/v4/Patient/17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
"resource": {
"resourceType": "Patient",
"id": "17b17befcde-8fb6aca7-9008-4682-ae7f-4c3c5a91c7e2",
"meta": {
"versionId": "1",
"lastUpdated": "2021-08-05T19:16:37.89885Z"
},
"text": {
"status": "generated",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\">Generated by <a href=\"https://github.com/synthetichealth/synthea\">Synthea</a>.Version identifier: master-branch-latest-2-g8e7e92c\n . Person seed: 7105486291024734541 Population seed: 0</div>"
},
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/patient-mothersMaidenName",
"valueString": "Graciela518 Badillo851"
},
{
"url": "http://hl7.org/fhir/StructureDefinition/patient-birthPlace",
"valueAddress": {
"city": "Caguas",
"state": "Puerto Rico",
"country": "PR"
}
},
{
"url": "http://synthetichealth.github.io/synthea/disability-adjusted-life-years",
"valueDecimal": 1.6165934938317164
},
{
"url": "http://synthetichealth.github.io/synthea/quality-adjusted-life-years",
"valueDecimal": 47.383406506168288
},
{
"url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex",
"valueCode": "F"
},
{
"url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex",
"valueCode": "M"
}
],
"identifier": [
{
"system": "https://github.com/synthetichealth/synthea",
"value": "bd958c64-56b6-f206-b03d-8f4b8a417215"
},
{
"type": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0203",
"code": "MR",
"display": "Medical Record Number"
}
],
"text": "Medical Record Number"
},
"system": "http://hospital.smarthealthit.org",
"value": "bd958c64-56b6-f206-b03d-8f4b8a417215"
},
{
"type": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0203",
"code": "SS",
"display": "Social Security Number"
}
],
"text": "Social Security Number"
},
"system": "http://hl7.org/fhir/sid/us-ssn",
"value": "999-93-6090"
},
}
]
References
