Question: How do I decode the Job Id from the Bulk Data Export or Import on the IBM FHIR Server?
You can use the following code to walk through and decode your job id using your passowrd.
Question: How do I decode the Job Id from the Bulk Data Export or Import on the IBM FHIR Server?
You can use the following code to walk through and decode your job id using your passowrd.
The IBM FHIR Server provides an eventing service that notifies about persistence events – CUD (Create-Update-Delete). The notification service can trigger specific actions in a downstream application. You can configure these events to flow to Apache Kafka.
If you want to configure the IBM FHIR Server without a keystore, you can configure SASL_SSL like the below (notice there is no keystore specified).
{
"fhirServer":{
"notifications":{
"kafka": {
"enabled": true,
"topicName": "FHIR_NOTIFICATIONS",
"connectionProperties": {
"bootstrap.servers": "broker-1.mybroker.com:9093,broker-2.mybroker.com:9093,broker-0.mybroker.com:9093,broker-5.mybroker.com:9093,broker-4.mybroker.com:9093,broker-3.mybroker.com:9093",
"security.protocol": "SASL_SSL",
"sasl.mechanism": "PLAIN",
"ssl.protocol": "TLSv1.2",
"ssl.enabled.protocols": "TLSv1.2",
"ssl.endpoint.identification.algorithm": "HTTPS",
"security.inter.broker.protocol": "SSL",
"sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"token\" password=\"MYPASSWORD\";",
"acks": "all",
"retries": "60",
"request.timeout.ms": "10000",
"max.block.ms": "60000",
"max.in.flight.requests.per.connection": "5",
"client.dns.lookup": "use_all_dns_ips"
}
}
}
}
}
If you run a workload against IBM FHIR Server, it’ll generate a set of notifications…
Download Kafka from https://kafka.apache.org
Create a client-ssl.properties
cat << EOF > client-ssl.properties
bootstrap.servers=broker-2.mybroker.com:9093,broker-5.mybroker.com:9093,broker-0.mybroker.com:9093,broker-3.mybroker.com:9093,broker-4.mybroker.com:9093,broker-1.mybroker.com:9093
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="MYPASSWORD";
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
ssl.protocol=TLSv1.2
EOF
Unzip the kafka archive
Check the Kafka Console Consumer
bash bin/kafka-console-consumer.sh --bootstrap-server broker-4.mybroker.com:9093,broker-5.mybroker.com:9093,broker-3.mybroker.com:9093,broker-2.mybroker.com:9093,broker-1.mybroker.com:9093,broker-0.mybroker.com:9093 --topic FHIR_NOTIFICATION --max-messages 25 --property print.timestamp=true --consumer.config client-ssl.properties
Yes, you can use the Batch API.
The Bundle.entry must include a request object pointing at URL <RESOURCE>/$validate with method POST.
The Resource that is validated must be wrapped in a Parameters object and added to the Parameters.parameter.resource
{
"resourceType": "Parameters",
"parameter": [
{
"name": "resource",
"resource": <COPY HERE>
}
]
}
Bundle.entry.resource, and make a request to the server root.curl --location --request POST 'https://localhost:9443/fhir-server/api/v4' \
--header 'Authorization: Basic ...' \
--header 'Content-Type: application/json' \
--data-raw '{
"resourceType": "Bundle",
"type": "transaction",
"entry": [
{
"fullUrl": "Patient/$validate",
"resource": {
"resourceType": "Parameters",
"parameter": [
{
"name": "resource",
"resource": {
"resourceType": "Patient",
"id": "example",
"name": [
{
"use": "official",
"family": "Chalmers",
"given": [
"Peter",
"James"
]
}
]
}
}
],
"meta": {
"tag": [
{
"system": "http://terminology.hl7.org/CodeSystem/v3-ActReason",
"code": "HTEST",
"display": "test health data"
}
]
}
},
"request": {
"method": "POST",
"url": "Patient/$validate"
}
}
]
}'
Bundle.entry.resource.response.issue to check the severity and details to see why the validation is failing.{
"resourceType": "Bundle",
"type": "transaction-response",
"entry": [
{
"resource": {
"resourceType": "OperationOutcome",
"id": "NoError",
"text": {
"status": "additional",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>No error</p></div>"
},
"issue": [
{
"severity": "warning",
"code": "invariant",
"details": {
"text": "dom-6: A resource should have narrative for robust management"
},
"expression": [
"Patient"
]
}
]
},
"response": {
"status": "200",
"outcome": {
"resourceType": "OperationOutcome",
"id": "NoError",
"text": {
"status": "additional",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>No error</p></div>"
},
"issue": [
{
"severity": "warning",
"code": "invariant",
"details": {
"text": "dom-6: A resource should have narrative for robust management"
},
"expression": [
"Patient"
]
}
]
}
}
}
]
}
You should add entries for each Resource you want to validate in the Batch.
In my GitHub Repository IBM FHIR Server, we use GitHub Actions to execute our Continuous Integration (CI) workflows. The CI workflows enable us to execute our code in complicated scenarios – a database (IBM Db2, Postgres), events system (Kafka (two zookeeper, two brokers), NATS) in a mix of various configurations.
I started to enable Docker Compose with Kafka and the IBM FHIR Server’s Audit module.
I ran into this error when running docker-compose exec in my workflow’s run.
TEST_CONFIGURATION: check that there is output and the configuration works
the input device is not a TTY
Error: Process completed with exit code 1.
It turns out this is well known in GitHub Actions https://github.com/actions/runner/issues/241#issuecomment-745902718 with a great example of a fix at https://github.com/gfx/example-github-actions-with-tty/blob/4c5f457c65dfe61e273e0470414699420be5e134/.github/workflows/test.yml#L18
I ended up changing my workflow, and it passed!
The key being in setting the shell value – shell: 'script -q -e -c "bash {0}"'
- name: Server Integration Tests - Audit
env:
WORKSPACE: ${{ github.workspace }}
shell: 'script -q -e -c "bash {0}"'
run: |
bash build/audit/bin/pre-integration-test.sh ${{matrix.audit}}
bash build/audit/bin/integration-test.sh ${{matrix.audit}}
bash build/audit/bin/post-integration-test.sh ${{matrix.audit}}
Hi Everyone.
Thanks for sitting down and watching this video. I’m going to show you how to quickly spin up a Docker image of IBM FHIR Server, check the logs, make sure it’s healthy, and how to use the fhir-examples module with the near search.
The following are the directions followed in the video:
Navigate to DockerHub: IBM FHIR Server
Run the Server docker run -p9443:9443 ibmcom/ibm-fhir-server
Note, startup may take 2 minutes as the image is bootstrapping a new Apache Derby database in the image. To use Postgres or IBM Db2, please review the documentation.
Review the docker logs
Check the server is up and operational
curl -k -i -u 'fhiruser:change-password' 'https://localhost:9443/fhir-server/api/v4/$healthcheck'
You now have a running IBM FHIR Sever.
Let’s load some data using a Jupyter Notebook.
The IBM FHIR Server team wraps specification and service unit tests into a module called fhir-examples and posts to Bintray: ibm-fhir-server-releases or go directly to the repository.
We’re going to use the python features and Jupyter Notebook to process the fhir-examples.
We’ll download the zip, filter the interesting jsons, and upload to the IBM FHIR Server in a loop.
entries = z.namelist()
for entry in entries:
if entry.startswith('json/ibm/bulk-data/location/'):
f = z.open(entry);
content = f.read()
r = requests.post('https://localhost:9443/fhir-server/api/v4/Location',
data=content,
headers=headers,
auth=httpAuth,
verify=False)
print('Done uploading - ' + entry)
We’re going to query the data on the IBM FHIR Server using the Search Query Parameter near to search within 10Km of Cambridge Massachusetts.
queryParams = {
'near': '42.373611|-71.110558|10|km',
"_count" : 200
}
Note, the IBM FHIR Server includes some additional search beyond the UCUM and WS48 units and it’s listed in at the Conformance page.
We’ll normalize this data and put in a Pandas dataframe.
From the dataframe, we can now add markers to the page.
cambridge = [ 42.373611, -71.11000]
map_cambridge_locs_from_server = folium.Map(location=cambridge, zoom_start=10)
# Iterate through the Rows
for location_row in location_rows :
# print(location_row)
# Cast the values into the appropriate types as FOLIUM will die weirdly without it.
lat_inc = float(location_row['resource.position.latitude'])
long_inc = float(location_row['resource.position.longitude'])
name_inc = str(location_row['resource.name'])
#print(lat_inc)
#print(long_inc)
#print(name_inc)
label = folium.Popup(name_inc, parse_html=True)
folium.CircleMarker(
[lat_inc, long_inc],
radius=5,
popup=label,
fill=True,
fill_color='red',
fill_opacity=0.7).add_to(map_cambridge_locs_from_server)
map_cambridge_locs_from_server
You can see the possibilities with the IBM FHIR Server and the near search.
Reference
The following takes the Capability Statements and generates the minimum coverage set of Resources across implementation guides based on the capability statements.
This is handy to check the number of profiles in use (beyond the base spec) and to know the profiles used on the server based on the capability statements.
cat capabilitystatements/*.json | jq -r '.rest[].resource[]| "|\(.type)|\(.supportedProfile)|"' | sort -u
| Resource | Profiles |
|---|---|
| AllergyIntolerance | http://hl7.org/fhir/us/core/StructureDefinition/us-core-allergyintolerance |
| CarePlan | http://hl7.org/fhir/us/core/StructureDefinition/us-core-careplan |
| CareTeam | http://hl7.org/fhir/us/core/StructureDefinition/us-core-careteam |
| Condition | http://hl7.org/fhir/us/core/StructureDefinition/us-core-condition |
| Device | http://hl7.org/fhir/us/core/StructureDefinition/us-core-implantable-device |
To check the operations rrequired in the implementation guides, you can use the following to process a set of capability statements into a useful outputs.
cat capabilitystatements/*.json | jq -r '.rest[].resource[]| "|\(.type)|\(.operation)|"' | grep -v null
| Resource | Operation | Conformance |
|---|---|---|
| ValueSet | $expand | SHOULD |
I hope this helps.
My team has been running more workloads on IBM Cloud, more specifically with IBM Db2. Our daily tools are slightly different to work with in the cloud, less administrative access and tools we can access on the host – db2batch, db2advis, db2expln and other native tools.
That’s when I ran across some great references that lead me in a direction that works for my team.
Login to the IBM Cloud console
Click Open Console

Expand Settings

Click Manage Users
Click Add

Click Add User
Enter the relevant details for the user
Click Create
I use my db2 docker container
Setup the SSL
mkdir -p /database/config/db2inst1/SSL_CLIENT
chmod -R 755 /database/config/db2inst1/SSL_CLIENT
/database/config/db2inst1/sqllib/gskit/bin/gsk8capicmd_64 -keydb \
-create -db "/database/config/db2inst1/SSL_CLIENT/ibmca.kdb" \
-pw "passw0rd" -stash
/database/config/db2inst1/sqllib/gskit/bin/gsk8capicmd_64 -cert \
-add -db "/database/config/db2inst1/SSL_CLIENT/ibmca.kdb" \
-pw "passw0rd" -file sqllib/cfg/DigiCertGlobalRootCA.arm
chmod 775 /database/config/db2inst1/SSL_CLIENT/ibmca.kdb
chmod 775 /database/config/db2inst1/SSL_CLIENT/ibmca.sth
Configure the database
db2 update dbm cfg using SSL_CLNT_KEYDB \
/database/config/db2inst1/SSL_CLIENT/ibmca.kdb
db2 update dbm cfg using SSL_CLNT_STASH
/database/config/db2inst1/SSL_CLIENT/ibmca.sth
db2 update dbm cfg using keystore_location
/database/config/db2inst1/SSL_CLIENT/ibmca.kdb
Restart the database
db2stop
db2start
Catalog the database
db2 catalog tcpip node cdtdb1 remote \
dashdb-txn-flex-yp-xxxx-xxxx.services.dal.bluemix.net server 50001 security ssl
db2 catalog db bludb as fhirblu4 at node cdtdb1
db2 connect to fhirblu4 user testpaul using ^PASSWORD^
If you have a problem connecting, log out of db2inst1 and log back in. It’ll activate the db2profile again.
Run db2expln
db2expln -d fhirblu4 -u testpaul "^PASSWORD^" -graph -f 1.sql \
-terminator ';' -o 1.out
Optimizer Plan:
Rows
Operator
(ID)
Cost
10
RETURN
( 1)
412211
|
10
TBSCAN
( 2)
412211
|
10
SORT
( 3)
412211
|
77909.3
HSJOIN
( 4)
412164
/------------------------/ \-----------------------\
88591.5 311638
TBSCAN TBSCAN
( 5) ( 8)
410438 1594.97
+---------------------------++--------------------------------+ |
354367 0.00309393 1 311638
Table: IXSCAN IXSCAN Table:
FHIRDATA2 ( 6) ( 7) FHIRDATA2
OBSERVATION_RESOURCES 7.52927 7.57425 OBSERVATION_LOGICAL_RESOURCES
| |
2.18146e+06 1.24649e+06
Index: Index:
FHIRDATA2 FHIRDATA2
IDX_OBSERVATION_TOKEN_VALUES_RPS IDX_OBSERVATION_STR_VALUES_RPS
Relevant References https://www.ibm.com/cloud/blog/how-to-use-an-api-key-or-access-token-to-connect-to-ibm-db2-on-cloud https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.admin.sec.doc/doc/t0053518.html https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11.1.0/com.ibm.db2.luw.admin.sec.doc/doc/c0070395.html https://developer.ibm.com/recipes/tutorials/ssl-how-to-configure-it-on-db2/ https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.admin.sec.doc/doc/t0012036.html https://www.ibm.com/support/knowledgecenter/en/SS6NHC/com.ibm.swg.im.dashdb.security.doc/doc/iam.html https://developer.ibm.com/recipes/tutorials/ssl-how-to-configure-it-on-db2/#r_step8
These are the containers settings for SSL:
db2inst1@4dda34a66a99 ~]$ db2 get dbm cfg | grep -i ssl
SSL server keydb file (SSL_SVR_KEYDB) = /database/config/db2inst1/SSL_CLIENT/ibmca.kdb
SSL server stash file (SSL_SVR_STASH) = /database/config/db2inst1/SSL_CLIENT/ibmca.sth
SSL server certificate label (SSL_SVR_LABEL) =
SSL service name (SSL_SVCENAME) =
SSL cipher specs (SSL_CIPHERSPECS) =
SSL versions (SSL_VERSIONS) =
SSL client keydb file (SSL_CLNT_KEYDB) = /database/config/db2inst1/SSL_CLIENT/ibmca.kdb
SSL client stash file (SSL_CLNT_STASH) = /database/config/db2inst1/SSL_CLIENT/ibmca.sth
Keystore location (KEYSTORE_LOCATION) = /database/config/db2inst1/SSL_CLIENT/ibmca.kdb
db2top -d fhirpdm -n pdmperf -u bluadmin -p password-removed
db2expln -d fhirdb -setup setup.sql -g -z \; -f uniq.sql -o plan.txt
json
{ "test" : { "test1" : "val1" } }
Open the JSON in vim and use python -m json.tool
:%!python -m json.tool
Results
json
{
"test": {
"test1": "val1"
}
}
Note: jq (jq is another option, but… not always available on every system, python tends to work everywhere).
References
Quick way to extract the main cert, and the intermediate CA and ROOT ca from a host.
shell
echo "" | openssl s_client -showcerts -prexit -connect HOSTNAME:443 2> /dev/null | sed -n -e '/BEGIN CERTIFICATE/,/END CERTIFICATE/ p'
One sees a PEM as output (just capture into a file one can use)
My Team just released IBM FHIR Server 4.2.2. Other than the amazing things documented and released with the release tab, I learned a few things.
If you need to replace tags, force it with fetch
~$ git fetch --tags -f
From github.com:IBM/FHIR
t [tag update] 4.1.0 -> 4.1.0
t [tag update] 4.2.2 -> 4.2.2
export BUILD_TYPE=RELEASE
export BUILD_VERSION=4.2.2
bash build/release/version.sh
mvn ${THREAD_COUNT} -ntp -B clean source:jar source:test-jar javadoc:jar \
install -f fhir-parent -Pfhir-validation-distribution,fhir-ig-carin-\
bb,fhir-ig-davinci-pdex-plan-net,fhir-ig-mcode,fhir-ig-us-core,deploy-\
bintray -DskipTests -pl ../fhir-ig-davinci-pdex-plan-net/,../fhir-\
validation -amd
-amd keeps the build focused only on the necessary packages (not the full fhir-parent)
su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRSERVER') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRSERVER'; END IF; END;\""
su - db2inst1 -c "db2 \"connect to fhirdb\" && db2 \" BEGIN IF (SELECT ROLENAME FROM SYSCAT.ROLES WHERE ROLENAME = 'FHIRBATCH') IS NULL THEN EXECUTE IMMEDIATE 'CREATE ROLE FHIRBATCH'; END IF; END;\""
Checking the Status of any command in a pipe, it was helpful in some automation where I had to wait on a jar to finish and check the output. Source
curl -L https://google.com | grep response | tee response.txt
RC=${PIPESTATUS[1]}
echo $RC
4