Cohort360-Backend serves as the backend of two main web applications: Cohort360 & Portail.
- Native JWT based authentication
- Support for OIDC authentication
- Support for external authentication services
- Role & Perimeter based access management
- Cohort360's requests versioning and cohort creation
- Real-time updates using web sockets
- Email-based notifications
- Manage data exports workflows; data can be exported in multiple formats: CSV, XLSX and Hive
- Web content management
- Toggle maintenance mode
- User impersonating for debugging
- Caching
- ...
🔹System
Cohort360-Backend allows to choose which apps to run by making them figure in INCLUDED_APPS. This uses Django's INSTALLED_APPS setting key.
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| DJANGO_SECRET_KEY | Assigned to Django's SECRET_KEY in settings.py. Learn more here |
yes |
|
| INCLUDED_APPS | Comma-separated apps names to consider by Django registry. | accesses,cohort_job_server,cohort,exporters,exports,content_management | no |
| DEBUG | Boolean to enable/disable debug mode. Make sure to set it to False in production | False | no |
| ADMINS | List of admin users to notify for errors. Used by Django's AdminEmailHandler. Multi-value variable ex: Admin1,[email protected];Admin2,[email protected] |
no | |
| NOTIFY_ADMINS | A boolean to allow sending error email notifications to ADMINS |
False | no |
| BACKEND_HOST | The backend host URL without the http scheme | localhost:8000 | no |
| FRONTEND_URL | Cohort360 frontend URL | http://local-cohort360.fr | no |
| FRONTEND_URLS | Comma-separated frontend URLs. if defined, it must include the FRONTEND_URL |
http://local-portail.fr,http://local-cohort360.fr | no |
| CELERY_BROKER_URL | Broker URL. Defaults to using redis | redis://localhost:6379 | no |
When Cohort360-Backend is running in multiprocessing mode (for example: using Gunicorn with workers), sending logs to a file may result in race conditions and logs loss as each process tries to write to the log file.
Refer to Python docs for more insights
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| SOCKET_LOGGER_HOST | Host URL to which the logs will be sent. Logs are currently sent to a Network SocketHandler |
localhost | no |
🔹Database
A PostgreSQL engine is used out of the box for Cohort360-Backend. Refer to DATABASES in settings.py to set up a different engine.
Learn more about available engines.
The following variables are required to define the default database connection:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| DB_HOST | localhost | yes |
|
| DB_PORT | 5432 | yes |
|
| DB_NAME | cohort_db | yes |
|
| DB_USER | cohort_dev | yes |
|
| DB_PASSWORD | cohort_dev_pwd | yes |
🔹Authentication
Cohort360-Backend uses Json Web Tokens to authenticate requests and requires these variables:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| JWT_ALGORITHMS | Comma-separated algorithms used to sign, encode and decode JWT tokens | RS256,HS256 | yes |
| JWT_SIGNING_KEY | Secret key used to sign JWT tokens | yes |
Two authentication modes are supported:
-
Authentication using credentials username/password (
default)
For this, Cohort360-Backend tries validating user credentials in the following order:- If there are third-party authentication services configured, try them.
- Otherwise, fall back to inplace authentication: check if the user is registered in database and the provided password is valid
An external authentication service may be for example an API that connects to an LDAP server under the hood.
💡 Refer to this annex for how to define hooks that use external authentication services -
OIDC authentication using one or multiple OIDC servers (
optional)
Activate this mode by settingENABLE_OIDC_AUTHto True (defaults to False)Note that Cohort360-Backend supports only
authorization_codegrant type for now
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| OIDC_AUTH_SERVER_1 | yes |
||
| OIDC_REDIRECT_URI_1 | yes |
||
| OIDC_CLIENT_ID_1 | yes |
||
| OIDC_CLIENT_SECRET_1 | yes |
||
| OIDC_AUDIENCE | comma-separated values of audience if multiple | yes |
|
| OIDC_EXTRA_SERVER_URLS | comma-separated URLs of other OIDC servers issuing tokens for users | no |
💡 Tip: You can configure a new server by adding extra variables:
OIDC_AUTH_SERVER_2,OIDC_REDIRECT_URI_2...
🔹Cohort
The Cohort app mainly allows to process Cohort Count and Cohort Creation requests
It interacts with a couple of FHIR and QueryExecutor instances to handle these requests in addition to a Solr
database being used behind the scenes.
- A Cohort Count request consist of getting the number of patients matching a set of criteria (i.e. get cohort size).
- A Cohort Creation request is about persisting the IDs of the patients included in a cohort for analysis or data export purposes.
Cohort360-Backend can manage creating cohorts according to the number of included patients:
- If the cohort size is within
COHORT_SIZE_LIMIT: it gets created within a few minutes - Otherwise, it can take up to 24h to be ready as the process of creating it involves data indexation in the Solr database.
TheSolr ETLentity then makes a callback to Cohort360-Backend to patch the cohort with its final status (created or failed)
The environment variables below are used to establish the architecture described before:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| FHIR_URL | The URL of the server used to translate FHIR criteria to Solr format, ex: https://fhir.aphp.fr | yes |
|
| QUERY_EXECUTOR_URL | ex: https://query-executor.aphp.fr. the URL of your instance of QueryExecutor | yes |
|
| QUERY_EXECUTOR_USERNAME | The system user of the QueryExecutor app. Used to make patch calls on Cohorts and Count Requests | yes |
|
| QUERY_EXECUTOR_TOKEN | Query Executor application API-key | yes |
|
| USE_SOLR | A boolean to indicate if a Solr database is used. If so, FHIR criteria are translated to Solr format | False | no |
| SOLR_ETL_USERNAME | The system user of the Solr ETL app. Used to make patch calls on Cohorts | yes if USE_SOLR |
|
| SOLR_ETL_TOKEN | ETL application API-key | yes if USE_SOLR |
|
| TEST_FHIR_QUERIES | Weather to test queries before sending them to QueryExecutor | False | no |
| LAST_COUNT_VALIDITY | Validity of a Count Request in hours. Passed this period, the request result becomes obsolete and the request must be re-executed | 24 | no |
| COHORT_SIZE_LIMIT | Maximum patients a "small" cohort can contain ("small" cohorts are created right away while big ones can take up to 24h) | 20000 | no |
🔹Perimeters
Perimeters are meant to represent entities and/or departments, within a health institution, where patients are treated.
They can be of different types (CHU, Hospital, Unit Service ...) and are organized in tree-like structure with a single parent perimeter (AP-HP for the Cohort360).
In Cohort360-Backend, perimeters are used for two main purposes:
- Grant user accesses: a user is assigned a specific Role over a Perimeter for a period of time.
- Run Cohort Requests: for example, fetch patients attached to one or multiple perimeters.
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| PERIMETER_TYPES | comma-separated types of perimeters | yes |
|
| ROOT_PERIMETER_ID | ID of the root (parent) perimeter | yes |
|
| REPORTING_PERIMETER_TYPES | comma-separated types of perimeters to include in the FeasibilityStudy report |
no |
If it is the case, perimeters can be synced using a periodic task.
For this, add accesses_perimeters to INCLUDED_APPS and set up a second database connection with the following variables:
A periodic task is provided in the accesses_perimeters app for this purpose.
cfSCHEDULED_TASKSfor configuration bellow.
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| PERIMETERS_SOURCE_DB_HOST | yes |
||
| PERIMETERS_SOURCE_DB_PORT | yes |
||
| PERIMETERS_SOURCE_DB_NAME | yes |
||
| PERIMETERS_SOURCE_DB_USER | yes |
||
| PERIMETERS_SOURCE_DB_PASSWORD | yes |
||
| PERIMETERS_SOURCE_DB_SCHEMAS | comma-separated schema names | public | no |
🔹Data Exports
If the exporters app is installed, the backend will allow to have data exported in the following formats:
- CSV files
- XLSX files
- To a Hive database
For this, the following variables are required:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| STORAGE_PROVIDERS | Comma-separated URLs of servers to store exported data | yes |
|
| EXPORT_API_URL | URL of the third-party API that handles exports | yes |
|
| EXPORT_API_AUTH_TOKEN | API-key used for authentication | yes |
|
| EXPORT_CSV_PATH | Path to the directory where CSV exports are stored | yes |
|
| EXPORT_XLSX_PATH | Path to the directory where XLSX exports are stored | yes |
|
| HADOOP_API_URL | URL of a third-party API that handles creating the database for Hive exports | yes |
|
| HADOOP_API_AUTH_TOKEN | API-key used for authentication | yes |
|
| HIVE_DB_PATH | Path to the directory where the Hive database is stored | yes |
|
| HIVE_USER | Name of the system user that creates Hive databases | yes |
|
| DISABLE_DATA_TRANSLATION | If True, it disables translating exported data. Export data as is | False | no |
| DAYS_TO_KEEP_EXPORTED_FILES | Number of days to keep exported data available for download | 7 | no |
🔹Scheduled tasks
Cohort360-Backend allows running scheduled tasks by providing task name, function to execute and a time of execution.
This uses Celery's Periodic tasks.
cf:
CELERY_BEAT_SCHEDULEin settings.py
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| SCHEDULED_TASKS | ; separated tasks configurations in the following format:task_name,module,function_name,hour,minutes |
no |
🔹Emailing system
To enable sending emails, Django's EmailBackend needs the following parameters:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| EMAIL_USE_TLS | True | yes |
|
| EMAIL_HOST | yes |
||
| EMAIL_PORT | yes |
||
| DEFAULT_FROM_EMAIL | Email address of sender | yes |
|
| EMAIL_SUPPORT_CONTACT | Email address to contact the support team | no |
🔹Email alerts for expiring accesses
Cohort360-Backend allows to frequently check users accesses validity and send email notifications for how to renew them.
To enable this behaviour, include the check_expiring_accesses task in SCHEDULED_TASKS.
By default, two emails are sent
30and2days respectively before accesses expiry date.
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| ACCESS_EXPIRY_FIRST_ALERT_IN_DAYS | Send first email X days before accesses expire | 30 | no |
| ACCESS_EXPIRY_SECOND_ALERT_IN_DAYS | Send second email X days before accesses expire | 2 | no |
| ACCESS_MANAGERS_LIST_LINK | A link to a file with the list of accesses managers | no |
🔹Misc
➖ Maintenances
Cohort360-Backend allows activating maintenance phase by restricting actions that change the state of the app.
With maintenance phase activated, only the following actions are allowed:
- READ actions (GET requests on all resources)
- Requests to
/auth/such as login, logout and refresh_token - Requests to
/maintenances/(to end the current maintenance phase for example)
For flexibility, the maintenance phase can be activated with two modes:
- Partial: as mentioned before, this allows to have the application running on read-only mode.
- Full: this make the app entirely out of service.
A cron job is configured to run by default every one minute to check if a maintenance phase is on progress. You can override this frequency by
modifying the MAINTENANCE_PERIODIC_SCHEDULING_MINUTES key in the settings.py file.
For the Cohort360 client, after each user login, a web socket connection is established allowing to send real-time updates regarding ongoing maintenances.
➖ Web content
The backend has a mini application called content_management useful to stream info messages and warnings to be displayed on the Cohort360
UI.
By adding it to the INCLUDED_APPS environment variable, new endpoints /webcontent/contents become available to handle basic CRUD operations.
➖ Regex
You can set optional regular expressions to validate usernames and email addresses
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| USERNAME_REGEX | A regex to validate usernames | (.*) | no |
| EMAIL_REGEX | A regex to validate email addresses | ^[\w.+-]+@[\w-]+.[\w]+$ | no |
➖ Caching responses
The backend uses RedisCache from the django_redis package.
Extra tuning parameters to control cache validity are defined in settings.py
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| ENABLE_CACHE | Enable caching HTTP responses | False | no |
➖ Store diagnosis data on InfluxDB
You can configure an InfluxDB connection to store response times of the API endpoints and plug in a monitoring tool like Grafana.
this activates a new middleware on top of the existing ones to track requests process time.
Start by adding the variable INFLUXDB_ENABLED set to True in addition to the following:
| Variable | Description | Default Value | Required ? |
|---|---|---|---|
| INFLUXDB_URL | InfluxDB server URL | yes |
|
| INFLUXDB_ORG | Organization name | yes |
|
| INFLUXDB_BUCKET | Bucket name | yes |
|
| INFLUXDB_DJANGO_TOKEN | InfluxDB API-key | yes |
git clone https://github.com/aphp/Cohort360-Back-end.git- Create a .env file in the admin_cohort directory following the .setup/.env.example template
🔆 More insights on environment variables above.
cp .setup/.env.example admin_cohort/.env- Create a perimeters.csv file in the .setup directory following the .setup/perimeters.example.csv format
cp .setup/perimeters.example.csv .setup/perimeters.csv🔶 Note that the setup.sh script needs root privileges to run some commands
bash .setup/setup.shAll set up 🎉
- The development server is running on port
8000and the Swagger UI for endpoints documentation is found at: localhost:8000/docs - Once on the Swagger UI, start by getting an authentication token by logging in using the
/auth/login/endpoint. Use that token for authorization.
See CONTRIBUTING.md for detailed process about contributing to this repository.
This project is licensed under the Apache License - see the LICENSE file for details.
If you find this project useful, please consider starring the repository and report any encountered bugs or issues.
Write to us at: [email protected]