Skip to content

jupyterhealth/jupyterhealth-exchange

BranchesTags

Repository files navigation

JupyterHealth Exchange

JupyterHealth Exchange is a Django web application that facilitates the sharing of user-consented medical data with authorized consumers through a web UI, REST and FHIR APIs.

In the context of JupyterHealth, data producers are typically study participants (FHIR Patients) using the CommonHealth Android App linked to personal devices (eg Glucose Monitors) and data consumers are typically researchers (FHIR Practitioners).

Features include:

Limitations & Status

This project is currently in a Proof of Concept stage, the project can be viewed on GitHub at the following URL:

https://github.com/orgs/the-commons-project/projects/8

Getting Started

Tip

If you are a user (Practitioner or Patient), please take note of the following access information for the JupyterHealth Website:

  • JHE Default Invite Code is helloworld (see here)
  • JHE Default Super User [email protected]/Jhe1234! can create and edit top level organizations. Any user can create sub-organizations.

Tip

Quick start: For local development, Skip the steps 8–12 as the seed_db command will register the Django OAuth2 application. Also, Pre‑generated values of OIDC_RSA_PRIVATE_KEY, PATIENT_AUTHORIZATION_CODE_CHALLENGE, and PATIENT_AUTHORIZATION_CODE_VERIFIER are provided in dot_env_example.txt for dev/demo use only.

Note

Due to browser security restrictions and the oidc-client-ts used for authentication, the web app must be accessed over HTTPS for any hostname other than localhost - see Running in Production below.

  1. Set up your Python environment and install dependencies from Pipfile - this project uses Django version 5.2 which requires python 3.10, 3.11, 3.12 or 3.13

    • NB: If using pipenv it is recommended to run pipenv sync against the lock file to match package versions

  2. Create a new Postgres DB (currently only Postgres is supported because of json functions)

  3. Copy dot_env_example.txt to .env and update the DB_* parameters to match (2) and generate a new value for SECRET_KEY, e.g. with openssl rand -base64 32.

  4. Ensure the .env is loaded into your Python environment, eg for pipenv run $ pipenv shell

  5. Run the Django migration $ python manage.py migrate to create the database tables.

  6. Seed the database by running the Django management command $ python manage.py seed

  7. Start the server with $ python manage.py runserver

    Skip steps 8-12 below if doing Quick start above

  8. Browse to http://localhost:8000/admin and enter the credentials [email protected] Jhe1234!

  9. Under Django OAuth Toolkit > Applications you should already see the seeded OAuth2 application (redirects include http://localhost:8000/auth/callback). Create a new application only if you need a custom client for testing or multi-tenant scenarios.

  10. Create an RS256 Private Key (step by step here)

  11. Create a new static PKCE verifier - a random alphanumeric string 44 chars long, and then create the challenge here.

  12. Return to the .env file

    • Update the OIDC_RSA_PRIVATE_KEY with the newly created Private Key
    • Update PATIENT_AUTHORIZATION_CODE_CHALLENGE and PATIENT_AUTHORIZATION_CODE_VERIFIER with PKCE static values generated above
    • Restart the python environment and Django server

  13. Browse to http://localhost:8000/ and log in with the credentials [email protected] Jhe1234! and you should be directed to the /portal/organizations path with some example Organizations is the dropdown

  14. Before the each commit always make sure to execute pre-commit run --all-files to make sure the PEP8 standards.

  15. Git hook for the pre-commit can also be installed pre-commit install to automate the process.

  16. If a hook fails, fix the issues, stage the changes, and commit again — the commit only succeeds when hooks pass.

Warning

The OIDC_RSA_PRIVATE_KEY, PATIENT_AUTHORIZATION_CODE_CHALLENGE, and PATIENT_AUTHORIZATION_CODE_VERIFIER provided in dot_env_example.txt are public demo keys for development only and must not be used in production.

Troubleshooting Local Development

Issue: Developers running Django on Windows (Visual Studio Code, Git Bash, etc.) might hit a blank screen after login even though authentication succeeds. That usually happens because the OIDC settings in settings.py get polluted with local paths (e.g. OIDC_CLIENT_REDIRECT_URI: http://localhost:8000C:/Program Files/Git/auth/callback).

Cause: The environment loader injects the shell’s current working directory or other path fragments into the OIDC URLs, producing malformed authorization/redirect addresses.

Solution: Explicitly define the two OIDC URLs in settings.py rather than relying on environment interpolation. For example:

OIDC_CLIENT_REDIRECT_URI = 'http://localhost:8000/auth/callback'
OIDC_CLIENT_AUTHORITY = 'http://localhost:8000/o/'

By hardcoding the values you prevent the path injection and keep the SPA from seeing broken URLs, which resolves the blank screen after login on Windows hosts.

Remote debugging tips

  1. Control log verbosity with DJANGO_LOG_LEVEL – The backend now reads the DJANGO_LOG_LEVEL environment variable (default: INFO) for both the root logger and the django logger. For Fly.io deployments, set DJANGO_LOG_LEVEL=DEBUG in fly.toml and run fly deploy. For local development, set DJANGO_LOG_LEVEL using the appropriate environment configuration for your operating system.

  2. Don’t forget the SPA debug banner – temporarily enabling Django DEBUG surfaces server-side tracebacks in the portal so you can correlate HTTP logs with richer context. Turn it off afterward to avoid exposing stack traces.

Note: DJANGO_LOG_LEVEL is a configuration flag, not a secret. Keep fly secrets and your .env strictly for secrets. This keeps local CI environments aligned with the same value.

Working with the Web UI

Patients & Practitioners

  • Any user accessing the Web UI is a Practitioner (data consumer) by default
  • Patient users (data producers) are registered by Practitioners and sent a link to authenticate and upload data
  • The same OAuth 2.0 strategy is used for both Practitioners and Patients, the only difference being that the authorization code is provided out-of-band for Patients (invitation link)

Organizations

  • An Organization is a group of Practitioners, Patients and Studies
  • An Organization is typically hierarchical with sub-Organizations eg Institution, Department, Lab etc
  • A Practitioner belongs to one or more Organization
  • A Patient belongs to one or more Organization
  • A Study belongs to one single Organization

Studies

  • A Study is a Group of Patients and belongs to a single Organization
  • A Study has one or more Data Sources and one or more Scope Requests
  • When a Patient is added to a Study, they must explicitly consent to sharing the requested Scopes before any data (Observations) can be uploaded or shared

Observations

  • An Observation is Patient data and belongs to a single Patient
  • An Observation must reference a Patient ID as the subject and a Data Source ID as the device
  • Personal device data is expected to be in the Open mHealth (JSON) format however the system can be easily extended to support any binary data attachments or discrete Observation records
  • Observation data is stored as a valueAttachment in Base 64 encoded JSON binary
  • Authorization to view Observations depends on the relationship of Organization, Study and Consents as described above

Data Sources

  • A Data Source is anything that produces Observations (typically a device app eg iHealth)
  • A Data Source supports one or more Scopes (types) of Observations (eg Blood Glucose)
  • An Observation references a Data Source ID in the device field

Front-end configuration hints for developers

  • The vanilla JavaScript SPA pulls runtime settings via core/templates/client/client_settings.js, which exposes a global CONSTANTS object.
  • CONSTANTS is populated by the Django context processor core/context_processors.py, so any new value you expose there becomes available both to the template and to core/static/js/client.js.
  • Keep secrets and URLs on the Django side (settings or database-backed values) and let the context processor synthesize them, to prevent duplication and to keep the SPA reusable across environments.

Use Case Example

  1. Sign up as a new user from the Web UI
  2. Create a new Organization (your user is automatically added to the Organization with a Manager role)
  3. Create a new Study for the Organization (View Organization > Studies+)
  4. Create a new Patient for the Organization using a different email than (1) (Patients > Add Patient)
  5. Add Data Sources and Scopes to the Study (View Study > Data Sources+, Scope Requests+)
  6. Add the Patient to the Study (Patients > check box > Add Patient(s) to Study)
  7. Create an Invitation Link for the Patient (View Patient > Generate Invitation Link)
  8. Use the code in the invitation link with the Auth API to swap it for an access token
  9. Upload Observations using the FHIR API and access token
  10. View the Observations from the web UI

Role Based Access Control

Whether or not a Practitioner user can view particular Patients, Studies and Observations depends on membership of the Organization that the Patients/Studies/Observations belong to.

Whether or not a Practitioner user can edit the Data Sources, Organization, Organization's Patients or Studies depends on the role they are assigned to for that Organization at the time of being added. When a user create a new Organization they are automatically added as a Manager of the Organization. Permissions for roles are outlined in the table below.

Permission Super User Manager Member Viewer
Edit Data Sources âś…
Edit Top Level Organizations âś…
Edit Organization âś… âś…
Edit Patients âś… âś… âś…
Edit Studies âś… âś… âś…
View All âś… âś… âś… âś…

Open mHealth Schemas

If Observations are sent with data attached in the Open mHealth format (eg Observation.code=omh:blood-glucose:4.0) JSON Schema validation is used to check the payload. Example values can be found at data/omh/examples/data-points and the JSON schemas can be found at data/omh/json-schemas. See Open mHealth for more information.

Test Data

Database Seed

  • The initial Database is seeded with a minimal set of records to provide an example of the different entity relationships, see the diagram below.
flowchart TD
    sam("SuperUser Sam<br/><small>sam\@example.com</small>")
    style sam fill:#CFC

    %% berkeley
    ucb("Organization:<br/>University of California Berkeley") --> ccdss("Organization:<br/>College of Computing, Data Science and Society")
    ccdss --> bids("Organization:<br/>Berkeley Institute for Data Science (BIDS)")

    %% berkeley users
    ucb --Manager--> mary("ManagerMary<br/><small>mary\@example.com</small>")
    style mary fill:#CFC
    ccdss --Manager--> mary
    bids --Manager--> mary
    bids --Memeber--> megan("MemberMegan<br/><small>megan\@example.com</small>")
    style megan fill:#CFC
    bids --Viewer--> victor("ViewerVictor<br/><small>victor\@example.com</small>")
    style victor fill:#CFC
    tom("ThreeOrgTom<br/><small>tom\@example.com</small>")
    bids --Viewer--> tom
    style tom fill:#CFC

    %% berkeley studies
    bids --> bidsStudyOnBPHR("BIDS Study on BP & HR<br/><small>Blood Pressure<br/>Heart Rate</small>")
    style bidsStudyOnBPHR fill:#CFF
    bids --> bidsStudyOnBP("BIDS Study on BP<br/><small>Blood Pressure </small>")
    style bidsStudyOnBP fill:#CFF

    %% berkeley patients
    bids --> peter("BidsPatientPeter<br/><small>peter\@example.com</small>")
    style peter fill:#FCC
    peter --Consented--> bidsStudyOnBPHR
    peter --Requested--> bidsStudyOnBP
    pamela("BidsPatientPamela<br/><small>pamela\@example.com</small>")
    style pamela fill:#FCC
    bids --> pamela
    pamela --Consented--> bidsStudyOnBPHR
    pamela --Consented--> bidsStudyOnBP

    %% ucsf
    ucsf("Organization:<br/>University of California San Francisco") --> med("Organization:<br/>Department of Medicine")
    med --> cardio("Organization:<br/>Cardiology")
    cardio --> moslehi("Organization:<br/>Moslehi Lab")
    cardio --> olgin("Organization:<br/>Olgin Lab")

    %% ucsf users
    ucsf --Manager-->mark("ManagerMark<br/><small>mark\@example.com</small>")
    style mark fill:#CFC
    med --Manager--> mark
    cardio --Manager--> mark
    moslehi --Member--> tom
    moslehi --Manager-->mark
    olgin --Manager--> tom

    %% ucsf studies
    cardio --> cardioStudyOnRR("Cardio Study on RR<br/><small>Respiratory rate</small>")
    style cardioStudyOnRR fill:#CFF
    moslehi --> moslehiStudyOnBT("Moslehi Study on BT<br/><small>Body Temperature</small>")
    style moslehiStudyOnBT fill:#CFF
    olgin --> olginStudyOnO2("Olgin Study on O2<br/><small>Oxygen Saturation</small>")
    style olginStudyOnO2 fill:#CFF

    %% ucsf patients
    moslehi --> percy("MoslehiPatientPercy<br/><small>percy\@example.com</small>")
    style percy fill:#FCC
    percy --Consented--> moslehiStudyOnBT
    olgin --> paul("OlginPatientPaul<br/><small>paul\@example.com</small>")
    style paul fill:#FCC
    paul --Consented--> olginStudyOnO2
    cardio --> pat("CardioOlginPatientPat<br/><small>pat\@example.com</small>")
    style pat fill:#FCC
    pat --Consented--> cardioStudyOnRR
    pat --Consented--> olginStudyOnO2
    olgin --> pat
Loading

Iglu Test Data

  • Additional test data from the iglu project can be seeded by running the following command (please note this can take 10-20 minutes to run) $ python manage.py iglu
  • This creates a new study under the "Berkeley Institute for Data Science (BIDS)" Organization with 19 mock patients and 1745 real Observation data points

Working with APIs

Auth API

  • The OAuth 2.0 Authorization Code grant flow with PKCE is used to issue Access, Refresh and ID tokens for both Practitioners (web login) and Patients (secret invitation link)
  • The Patient authorization code is generated by the server and then shared with the user out-of-band as a secret invitation link
  • OAuth is configured from the Django Admin page (See Getting Started above)
  • Endpoints and configuration details can be discovered from the OIDC metadata endpoint: /o/.well-known/openid-configuration
  • The returned Access Token should be included in the Authorization header for all API requests with the prefix Bearer
  • Because the Patient authorization code is generated by the server, the PKCE code challenge and code verifier for Patient auth must be static values and set by the env vars (example below). The Patient client then sends this code_verifier along with the authorization code to obtain tokens. The redirect_uri serves no purpose (as the initial authorization code has already been issued) but is required per OAuth spec.
PATIENT_AUTHORIZATION_CODE_CHALLENGE = '-2FUJ5UCa7NK9hZWS0bc0W9uJ-Zr_-Pngd4on69oxpU'
PATIENT_AUTHORIZATION_CODE_VERIFIER  = 'f28984eaebcf41d881223399fc8eab27eaa374a9a8134eb3a900a3b7c0e6feab5b427479f3284ebe9c15b698849b0de2'

Client POST
Content-Type: application/x-www-form-urlencoded
code=4AWKhgaaomTSf9PfwxN4ExnXjdSEqh&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fexample.com%2Fauth%2Fcallback
&client_id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
&code_verifier=f28984eaebcf41d881223399fc8eab27eaa374a9a8134eb3a900a3b7c0e6feab5b427479f3284ebe9c15b698849b0de2

Note

It is understood using static values for PKCE runs against best practises however this is only used for the Patient client auth and not the Practitioner Web UI or API auth. The Patient client authorization code is generated by the server and shared out of band and therefore dynamic PKCE can not be used unless it is passed along with the invitation secret link, which would defeat the purpose of an additional check.

Invitation Link Structure

An invitation link may look something like below

https://play.google.com/store/apps/details?id=org.thecommonsproject.android.phr.dev&referrer=cloud_sharing=jhe.fly.dev|LhS05iR1rOnpS4JWfP6GeVUIhaRcRh
  • The purpose of the prefix URL (eg https://play.google.com/store/apps/details?id=org.thecommonsproject.android.phr.dev&referrer=cloud_sharing= ) is for the device to know which app to launch. In this case, the CommonHealth app is launched via Google Play so that if the user does not yet have the CommonHealth app installed they can install it within the flow
  • The prefix URL component may be more simple, for example https://carex.ai/?invitation= to launch the CareX app
  • The suffix of the link contains the hostname (optional) followed by a pipe character and the OAuth2 Authorization Code, for example jhe.fly.dev|LhS05iR1rOnpS4JWfP6GeVUIhaRcRh
  • The purpose of the suffix is to provide the app with information on what host to talk to (as there may be many JHEs configured for the one Patient) as well as the Authorization Code that can be swapped for an Access Token to use the API (see above)
  • The prefix URL is configured in the .env (and dot_env_example.txt) as CH_INVITATION_LINK_PREFIX; the example value points to the CommonHealth Play Store deep link shown above.
  • The host name is included by default but can optionally be removed from the link (if there will only ever be one host for the app) by configuring the .env with CH_INVITATION_LINK_EXCLUDE_HOST=True
  • So in the example of https://carex.ai/?invitation=jhe.fly.dev|LhS05iR1rOnpS4JWfP6GeVUIhaRcRh
    1. The CareX app is launched with the URL
    2. The CareX app parses the invitation parameter
    3. The CareX app gets the token endpoint from the invitation host https://jhe.fly.dev/o/.well-known/openid-configuration
    4. The CareX app posts LhS05iR1rOnpS4JWfP6GeVUIhaRcRh to get an access token
    5. The CareX app uses the API below to set consents
    6. The CareX app uses the API below to upload data

Single Sign-On (SSO) with SAML2

The django-saml2-auth library is included to support SSO with SAML2.

Example SAML2 Flow with mocksaml.com

Modify the .env to match below

# Default: SSO disabled. Change to 1 to enable.
SAML2_ENABLED=1

# Comma-separated list matches email domains permitted to sign in via SSO
SSO_VALID_DOMAINS=example.com,example.org

# MockSAML metadata URL (used to auto-configure IdP endpoints & certificate)
IDENTITY_PROVIDER_METADATA_URL=https://mocksaml.com/api/saml/metadata
Temporarily Switch on Debug for logging

Add the below to ./settings.py

# settings.py (demo only)
DEBUG = True

Warning

Use Debug for testing only, switch off Debug for Production.

When DEBUG is enabled the SPA debug page now summarizes server errors (including HTML tracebacks), auto-scrolls the banner into view, and auto-hides after a few seconds so developers can quickly see the actionable message.

Test Flow

  1. Visit https://mocksaml.com/saml/login and enter the fields below:

    • ACS URL: http://localhost:8000/sso/acs/ (or substitute your hostname) End with a trailing slash

    • Audience: http://localhost:8000/sso/acs/

  2. Enter any email name @example.com

  3. Enter any password

  4. Click Sign in

  5. The JHE portal should be displayed with the user in the matching user name in the bottom left hand corner

Notes
  • The JHE app reads IdP configuration from IDENTITY_PROVIDER_METADATA_URL
  • Only users with email addresses on the SSO_VALID_DOMAINS are permitted

Admin REST API

  • The Admin API is used by the Web UI SPA for Practitioner/Patient/Organization/Study management and Patient data provider apps/clients to manage Patient consents.

Profile

  • The profile endpoint returns the current user details.
// GET /api/v1/users/profile
{
    "id": 10001,
    "email": "[email protected]",
    "firstName": "Peter",
    "lastName": "ThePatient",
    "patient": {
        "id": 40001,
      	...
    }
}

Patient Consents

  • The consents endpoint returns the studies that are pending and consented for the specified Patient. In this example, the Patient has been invited to Demo Study 2 and has already consented to sharing blood glucose data with Demo Study 1.
// GET /api/v1/patients/40001/consents
{
    "patient": {
        "id": 40001,
				//...
    },
    "consolidatedConsentedScopes": [
        {
            "id": 50002,
            "codingSystem": "https://w3id.org/openmhealth",
            "codingCode": "omh:blood-pressure:4.0",
            "text": "Blood pressure"
        }
    ],
    "studiesPendingConsent": [
        {
            "id": 30002,
            "name": "Demo Study 2",
            "organization": { ... }
            "dataSources": [ ... ],
            "pendingScopeConsents": [
                {
                    "code": {
                        "id": 50002,
                        "codingSystem": "https://w3id.org/openmhealth",
          							"codingCode": "omh:blood-pressure:4.0",
                        "text": "Blood pressure"
                    },
                    "consented": null
                }
            ]
        }
    ],
    "studies": [
        {
            "id": 30001,
            "name": "Demo Study 1",
            "organization": { ... },
            "dataSources": [ ... ],
            "scopeConsents": [
                {
                    "code": {
                        "id": 50001,
                        "codingSystem": "https://w3id.org/openmhealth",
                        "codingCode": "omh:blood-glucose:4.0",
                        "text": "Blood glucose"
                    },
                    "consented": true
                }
            ]
        }
    ]
}
  • To respond to requested consents, a POST is sent to the same consents endpoint with the scope and the consented boolean.
// POST /api/v1/patients/40001/consents
{
  "studyScopeConsents": [
    {
      "studyId": 30002,
      "scopeConsents": [
        {
            "codingSystem": "https://w3id.org/openmhealth",
            "codingCode": "omh:blood-pressure:4.0",
            "consented": true
        }
      ]
    }
  ]
}
  • A PATCH request can be sent with the same payload to update an existing Consent
  • A DELETE request can be sen with the same payload excluding scopeConsents.consented to delete the Consent

FHIR REST API

Patients

  • The FHIR Patient endpoint returns a list of Patients as a FHIR Bundle for a given Study ID passed as query parameter_has:Group:member:_id or alternatively a single Patient matching the query parameter identifier=<system>|<value>
Query Parameter Example Description
_has:Group:member:_id 30001 Filter by Patients that are in the Study with ID 30001
identifier `http://ehr.example.com abc123` 
// GET /fhir/r5/Patient?_has:Group:member:_id=30001
{
    "resourceType": "Bundle",
    "type": "searchset",
    "entry": [
        {
            "resource": {
                "resourceType": "Patient",
                "id": "40001",
                "meta": {
                    "lastUpdated": "2024-10-23T12:35:25.142027+00:00"
                },
                "identifier": [
                    {
                        "value": "fhir-1234",
                        "system": "http://ehr.example.com"
                    }
                ],
                "name": [
                    {
                        "given": [
                            "Peter"
                        ],
                        "family": "ThePatient"
                    }
                ],
                "birthDate": "1980-01-01",
                "telecom": [
                    {
                        "value": "[email protected]",
                        "system": "email"
                    },
                    {
                        "value": "347-111-1111",
                        "system": "phone"
                    }
                ]
            }
        },
        ...

Observations

  • The FHIR Observation endpoint returns a list of Observations as a FHIR Bundle
  • At least one of Study ID, passed as patient._has:Group:member:_id or Patient ID, passed as patient or Patient Identifier passed as patient.identifier=<system>|<value> query parameters are required
  • subject.reference references a Patient ID
  • device.reference references a Data Source ID
  • valueAttachment is Base 64 Encoded Binary JSON
Query Parameter Example Description
patient._has:Group:member:_id 30001 Filter by Patients that are in the Study with ID 30001
patient 40001 Filter by single Patient with ID 40001
patient.identifier `http://ehr.example.com abc123`
code `https://w3id.org/openmhealth omh:blood-pressure:4.0` 
// GET /fhir/r5/Observation?patient._has:Group:member:_id=30001&patient=40001&code=https://w3id.org/openmhealth|omh:blood-pressure:4.0
{
    "resourceType": "Bundle",
    "type": "searchset",
    "entry": [
        {
            "resource": {
                "resourceType": "Observation",
                "id": "63416",
                "meta": {
                    "lastUpdated": "2024-10-25T21:14:02.871132+00:00"
                },
                "identifier": [
                    {
                        "value": "6e3db887-4a20-3222-9998-2972af6fb091",
                        "system": "https://ehr.example.com"
                    }
                ],
                "status": "final",
                "subject": {
                    "reference": "Patient/40001"
                },
                "device": {
                  "reference": "Device/70001"
                },
                "code": {
                    "coding": [
                        {
                            "code": "omh:blood-pressure:4.0",
                            "system": "https://w3id.org/openmhealth"
                        }
                    ]
                },
                "valueAttachment": {
                    "data": "eyJib2R5IjogeyJlZmZlY3RpdmVfdGltZV9mcmFtZSI6IHsiZGF0ZV90aW1lIjogIjIwMjQtMDUt\nMDJUMDc6MjE6MDAtMDc6MDAifSwgInN5c3RvbGljX2Jsb29kX3ByZXNzdXJlIjogeyJ1bml0Ijog\nIm1tSGciLCAidmFsdWUiOiAxMjJ9LCAiZGlhc3RvbGljX2Jsb29kX3ByZXNzdXJlIjogeyJ1bml0\nIjogIm1tSGciLCAidmFsdWUiOiA3N319LCAiaGVhZGVyIjogeyJ1dWlkIjogIjZlM2RiODg3LTRh\nMjAtMzIyMi05OTk4LTI5NzJhZjZmYjA5MSIsICJtb2RhbGl0eSI6ICJzZW5zZWQiLCAic2NoZW1h\nX2lkIjogeyJuYW1lIjogImJsb29kLXByZXNzdXJlIiwgInZlcnNpb24iOiAiMy4xIiwgIm5hbWVz\ncGFjZSI6ICJvbWgifSwgImNyZWF0aW9uX2RhdGVfdGltZSI6ICIyMDI0LTEwLTI1VDIxOjEzOjMx\nLjQzOFoiLCAiZXh0ZXJuYWxfZGF0YXNoZWV0cyI6IFt7ImRhdGFzaGVldF90eXBlIjogIm1hbnVm\nYWN0dXJlciIsICJkYXRhc2hlZXRfcmVmZXJlbmNlIjogImh0dHBzOi8vaWhlYWx0aGxhYnMuY29t\nL3Byb2R1Y3RzIn1dLCAic291cmNlX2RhdGFfcG9pbnRfaWQiOiAiZTZjMTliMDQyOGM4NWJiYjdj\nMTk4MGNiOTRkZDE3N2YiLCAic291cmNlX2NyZWF0aW9uX2RhdGVfdGltZSI6ICIyMDI0LTA1LTAy\nVDA3OjIxOjAwLTA3OjAwIn19",
                    "contentType": "application/json"
                }
            }
        },
        ...
  • Observations are uploaded as FHIR Batch bundles sent as a POST to the root endpoint
// POST /fhir/r5/
{
  "resourceType": "Bundle",
  "type": "batch",
  "entry": [
    {
      "resource": {
        "resourceType": "Observation",
        "status": "final",
        "code": {
          "coding": [
            {
              "system": "https://w3id.org/openmhealth",
              "code": "omh:blood-pressure:4.0"
            }
          ]
        },
        "subject": {
          "reference": "Patient/40001"
        },
        "device": {
          "reference": "Device/70001"
        },
        "identifier": [
            {
                "value": "6e3db887-4a20-3222-9998-2972af6fb091",
                "system": "https://ehr.example.com"
            }
        ],
        "valueAttachment": {
          "contentType": "application/json",
          "data": "eyJzeXN0b2xpY19ibG9vZF9wcmVzc3VyZSI6eyJ2YWx1ZSI6MTQyLCJ1bml0IjoibW1IZyJ9LCJkaWFzdG9saWNfYmxvb2RfcHJlc3N1cmUiOnsidmFsdWUiOjg5LCJ1bml0IjoibW1IZyJ9LCJlZmZlY3RpdmVfdGltZV9mcmFtZSI6eyJkYXRlX3RpbWUiOiIyMDIxLTAzLTE0VDA5OjI1OjAwLTA3OjAwIn19"
        }
      },
      "request": {
        "method": "POST",
        "url": "Observation"
      }
    },
    ...

Practitioner Script: Upload Observations for Authorized Patients

Prerequisites

  • The database must already be seeded
  • The Django server must be running, i.e., https://jhe.fly.dev or http://localhost:
  • The target Practitioner, Organization, and Study records already exist

Test Procedure

  1. Select Parent Organization Choose University of California, Berkeley.

  2. Open Sub-Organization Click View for Berkeley Institute for Data Science (BIDS).

  3. Create a New Study

    • Under BIDS, create a study.
    • Add the iHealth data source.
    • Set the data scope to blood glucose.

  4. Record the Study ID Open the newly created study and copy its Study ID (e.g. 30006).

  5. Run the Practitioner Upload Script Replace <study_id> with the ID from step 4:

    python resources/practitioner_fhir_obs_upload.py \
  --email [email protected] \
  --study-id <study_id>

Usage

python resources/practitioner_fhir_obs_upload.py \
  [--email <practitioner_email>] \
  [--password <practitioner_password>] \
  [--org-id <organization_id>] \
  [--study-id <study_id>] \
  [--patient-email <patient_email>]

Arguments & Defaults

Flag Description Default
--email Practitioner login email [email protected]
--password Practitioner password Jhe1234!
--org-id Target organization ID 20003
--study-id Target study ID (uses iHealth data source + blood-glucose scope) 30006
--patient-email Patient email (lookup / create / enroll) [email protected]

If omitted, the defaults will be used. You may override any or all flags.

Examples

With all defaults:

python practitioner_fhir_obs_upload.py

Specifying all arguments:

python practitioner_fhir_obs_upload.py \
  --email [email protected] \
  --password "Sup3r$ecret!" \
  --org-id 42 \
  --study-id 99 \
  --patient-email [email protected]

Running in Production

Django Server

The Django development server should not be used for production - more information is available at the official Django Deployment docs. One option included with JHE is the gunicorn server with WhiteNoise for static files, which can be run with the commands below.

$ python manage.py collectstatic --no-input
$ gunicorn --bind :8000 --workers 2 jhe.wsgi

HTTPS

Due to browser security restrictions and the oidc-client-ts used for authentication, the web app must be accessed over HTTPS for any hostname other than localhost. Below is an example of serving the app over HTTPS using an NGINX reverse proxy.

Install nginx and certbot
$ sudo apt update
$ sudo apt install -y nginx certbot

For servers that are publicly reachable

1. Configure nginx
$ DOMAIN=YOUR_DOMAIN
$ sudo mkdir -p /var/www/certbot

$ sudo tee /etc/nginx/sites-available/jhe.conf >/dev/null <<NGINX
upstream jhe_app { server 127.0.0.1:8000; keepalive 16; }

server {
  listen 80;
  server_name ${DOMAIN};

  # ACME HTTP-01
  location ^~ /.well-known/acme-challenge/ {
    root /var/www/certbot;
  }

  # everything else HTTPS
  location / { return 301 https://\$host\$request_uri; }
}

server {
  listen 443 ssl http2;
  server_name ${DOMAIN};

  # temp self-signed while we fetch LE cert (optional if you already have none)
  ssl_certificate     /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;

  client_max_body_size 25m;

  location / {
    proxy_pass http://jhe_app;
    proxy_http_version 1.1;
    proxy_set_header Host               \$host;
    proxy_set_header X-Real-IP          \$remote_addr;
    proxy_set_header X-Forwarded-For    \$proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto  https;
    proxy_read_timeout 300s;
    proxy_send_timeout 300s;
  }
}
NGINX

$ sudo ln -sf /etc/nginx/sites-available/jhe.conf /etc/nginx/sites-enabled/jhe.conf
$ sudo rm -f /etc/nginx/sites-enabled/default
$ sudo nginx -t && (sudo nginx -s reload || sudo nginx)
2. Get the certificate
$ sudo certbot certonly --webroot -w /var/www/certbot -d "$DOMAIN" --agree-tos -m admin@"$DOMAIN" --no-eff-email
3. Update NGINX config to point to the new certificate and reload
$ sudo sed -i "s#ssl-cert-snakeoil.pem#/etc/letsencrypt/live/$DOMAIN/fullchain.pem#g; s#ssl-cert-snakeoil.key#/etc/letsencrypt/live/$DOMAIN/privkey.pem#g" /etc/nginx/sites-available/jhe.conf
sudo nginx -t && sudo nginx -s reload
4. Auto-renew (no systemd needed)
echo '0 3 * * * root certbot renew --quiet --post-hook "nginx -s reload"' | sudo tee /etc/cron.d/certbot_renew >/dev/null

For servers that are not publicly reachable (DNS-01)

1. Get the certificate
$ DOMAIN=YOUR_DOMAIN

# issue with manual DNS challenge
$ sudo certbot certonly --manual --preferred-challenges dns -d "$DOMAIN" --agree-tos -m admin@"$DOMAIN" --no-eff-email
# follow prompts to add the _acme-challenge TXT record, wait for DNS to propagate, then continue
2. Configure NGINX and reload
$ sudo tee /etc/nginx/sites-available/jhe.conf >/dev/null <<NGINX
upstream jhe_app { server 127.0.0.1:8000; keepalive 16; }

server {
  listen 80;
  server_name ${DOMAIN};
  return 301 https://\$host\$request_uri;
}

server {
  listen 443 ssl http2;
  server_name ${DOMAIN};

  ssl_certificate     /etc/letsencrypt/live/${DOMAIN}/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/${DOMAIN}/privkey.pem;

  client_max_body_size 25m;

  location / {
    proxy_pass http://jhe_app;
    proxy_http_version 1.1;
    proxy_set_header Host               \$host;
    proxy_set_header X-Real-IP          \$remote_addr;
    proxy_set_header X-Forwarded-For    \$proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto  https;
    proxy_read_timeout 300s;
    proxy_send_timeout 300s;
  }
}
NGINX

$ sudo ln -sf /etc/nginx/sites-available/jhe.conf /etc/nginx/sites-enabled/jhe.conf
$ sudo rm -f /etc/nginx/sites-enabled/default
$ sudo nginx -t && (sudo nginx -s reload || sudo nginx)
3. Renewals

For hands-off renewals with DNS-01, use a Certbot DNS plugin for your provider (e.g., python3-certbot-dns-cloudflare, python3-certbot-dns-route53) and issue with that plugin once then renewals become automatic. An example for Cloudflare is below.

$ sudo apt install -y python3-certbot-dns-cloudflare
$ sudo tee /root/cf.ini >/dev/null <<EOF
dns_cloudflare_api_token = <TOKEN_WITH_DNS_EDIT>
EOF

$ sudo chmod 600 /root/cf.ini
$ sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials /root/cf.ini -d "$DOMAIN"

Architecture

Django

Django is a mature and well-supported web framework but was specifically chosen due to resourcing requirements. There are a few accommodations that had to be made for Django to support FHIR as described below.

camelCase

  • FHIR uses camelCase whereas Django uses snake_case.
  • The djangorestframework-camel-case library is used to support camelCase but the conversion happens downstream whereas the schema validation happens upstream, so manually calling humps is also required in parts.

DRF Serializers and Pydantic

  • The Django Rest Framework uses the concept of Serializers to validate schemas, whereas the FHIR validator uses Pydantic.

  • It is not reasonable to re-write the entire validation in the Serializer, so instead a combination of the two are used:

    • Top-level fields (most importantly the id of a record) are managed by the Serializer.
    • Nested fields (for example code{}.coding[].system above) are configured as a JSON field in the Serializer (so the top level field is this example is code) and then Pydantic is used to validate the whole schema including nested JSON.

  • There is a library that may allow Pydantic to be used as a Serializer but this needs to be explored further

JSON Responses

  • Postgres has rich JSON support allowing responses to be built directly from a raw Django SQL queries rather than using another layer of transforming logic.

Single Page App (SPA) Web UI

A hard requirement was to avoid additional servers and frameworks (eg npm, react, etc) for the front end Web UI. Django supports traditional server-side templating but a modern Single Page App is better suited to this use case of interacting with the Admin REST API. For these reasons, a simple Vanilla JS SPA has been developed using handlebars to render client side views from static HTML served using Django templates. The only other additional dependencies are oidc-clinet-ts for auth and bootstrap for styling.

Data Model

erDiagram
  "JheUser (FHIR Person)" {
    int id PK
    string email UK
    boolean email_is_verified
    string identifier
    string user_type
  }

  "Organization (FHIR Organization)" {
    int id PK
    string name
    string type
    int part_of_id FK
  }

  "Practitioner (FHIR Practitioner)" {
    int id PK
    int jhe_user_id FK
    string identifier
    string name_family
    string name_given
    date birth_date
    string telecom_phone
    datetime last_updated
  }

  "Patient (FHIR Patient)" {
    int id PK
    int jhe_user_id FK
    string identifier
    string name_family
    string name_given
    date birth_date
    string telecom_phone
    datetime last_updated
  }

  PractitionerOrganization {
    int id PK
    int practitioner_id FK
    int organization_id FK
    string role
  }

  PatientOrganization {
    int id PK
    int patient_id FK
    int organization_id FK
  }

  "CodeableConcept (FHIR CodeableConcept)" {
    int id PK
    string coding_system
    string coding_code
    string text
  }

  "Study (FHIR Group)" {
    int id PK
    string name
    string description
    int organization_id FK
    string icon_url
  }

  StudyPatient {
    int id PK
    int study_id FK
    int patient_id FK
  }

  StudyPatientScopeConsent {
    int id PK
    int study_patient_id FK
    string scope_actions
    int scope_code_id FK
    boolean consented
    datetime consented_time
  }

  StudyScopeRequest {
    int id PK
    int study_id FK
    string scope_actions
    int scope_code_id FK
  }

  DataSource {
    int id PK
    string name
    string type
  }

  DataSourceSupportedScope {
    int id PK
    int data_source_id FK
    int scope_code_id FK
  }

  StudyDataSource {
    int id PK
    int study_id FK
    int data_source_id FK
  }

  "Observation (FHIR Observation)" {
    int id PK
    int subject_patient_id FK
    int codeable_concept_id FK
    int data_source_id FK
    string value_attachment_data
    datetime last_updated
    string status
  }

  ObservationIdentifier {
    int id PK
    int observation_id FK
    string system
    string value
  }

  "Organization (FHIR Organization)" ||--o{ "Organization (FHIR Organization)" : _
  "JheUser (FHIR Person)" ||--|| "Practitioner (FHIR Practitioner)" : _
  "JheUser (FHIR Person)" ||--|| "Patient (FHIR Patient)" : _

  "Practitioner (FHIR Practitioner)" ||--o{ PractitionerOrganization : _
  "Organization (FHIR Organization)" ||--o{ PractitionerOrganization : _
  "Patient (FHIR Patient)" ||--o{ PatientOrganization : _
  "Organization (FHIR Organization)" ||--o{ PatientOrganization : _

  "Organization (FHIR Organization)" ||--o{ "Study (FHIR Group)" : _
  "Study (FHIR Group)" ||--o{ StudyPatient : _
  "Patient (FHIR Patient)" ||--o{ StudyPatient : _

  StudyPatient ||--o{ StudyPatientScopeConsent : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ StudyPatientScopeConsent : _

  "Study (FHIR Group)" ||--o{ StudyScopeRequest : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ StudyScopeRequest : _

  DataSource ||--o{ DataSourceSupportedScope : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ DataSourceSupportedScope : _

  "Study (FHIR Group)" ||--o{ StudyDataSource : _
  DataSource ||--o{ StudyDataSource : _

  "Patient (FHIR Patient)" ||--o{ "Observation (FHIR Observation)" : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ "Observation (FHIR Observation)" : _
  DataSource ||--o{ "Observation (FHIR Observation)" : _

  "Observation (FHIR Observation)" ||--o{ ObservationIdentifier : _
Loading

Deployment

For deployment options and a comprehensive guide take a look at the official Django Deployment docs

Deploying with the published container image

An example Dockerfile is included to deploy the app using gunicorn and WhiteNoise for static files. This image is published to ghcr.io/jupyterhealth/jupyterhealth-exchange.

  1. Create a new empty Postgres database
  2. Copy dot_env_example.txt to .env and update the DB_* parameters from (1) and generate a new value for SECRET_KEY, e.g. with openssl rand -base64 32.
  3. start a container with the image, mounting your .env file, with 
    TAG=sha-abc1234
docker run --rm -it -v$PWD/.env:/code/.env ghcr.io/jupyterhealth/jupyterhealth-exchange:$TAG bash
  4. Migrate the DB by running python manage.py migrate
  5. Seed the database by running the Django management command python manage.py seed_db
  6. exit your setup container and launch a new container running the JupyterHealth Exchange 
    docker run -v$PWD/.env:/code/.env -p8000:8000 ghcr.io/jupyterhealth/jupyterhealth-exchange:$TAG

About

Django web app for sharing clinical data over a FHIR API

jupyterhealth.org

Topics

django fhir identity-provider fhir-server clinical-data-management medical-research

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

8 stars

Watchers

5 watching

Forks

7 forks
Report repository

Releases

8 tags

Packages

 
 
 

Contributors 8

Languages