Formalise the process of data access, processing and creation of ResourceFile

[tbhallett]

In order to make the TLO model be keep up-to-date for Malawi, and to make application to other countries as easy as possible, we need to formalise the process we have gone through already of data access, processing and creation of the ResourceFiles.

The follows types of data are suitable for this:

• Population Data: @tbhallett assembles the necessary population data used in the model is here and the scripting file that generate them is here

• Global Burden of Disease: @tbhallett has processed file of the GBD results (downloadable from https://vizhub.healthdata.org) using this script to generate ResourceFile that are used in the simulation to determine the risk of course of death, and in calibration of the model (https://github.com/UCL/TLOmodel/blob/master/src/scripts/data_file_processing/formatting_gbd_data.py)

• Fertility and Contraception Use Data: @timcolbourn developed an analysis of the DHS data to estimate schedules of fertility and contraception uses in the model https://github.com/UCL/TLOmodel/tree/master/resources/contraception (This could probably be simplified if we didn't want to capture the process of switching contraceptive methods so comprehensively)

• Facilities Data: tbhallett and @BinglingICL used data provided by Malawi from its master facilities list to generate an index of all the types of facilities to represent in the model): https://github.com/UCL/TLOmodel/blob/master/resources/healthsystem/organisation/ResourceFile_Master_Facilities_List.csv

• Human Resources Data: @BinglingICL has developed scripts that take in several CHAI Workforce Optimisation Model datasets to generate ResourceFile that describe:

  • The types of appointment that are possible and the time that each takes (https://github.com/UCL/TLOmodel/tree/master/resources/healthsystem/human_resources/definitions)
  • The amount of patient-facing time available from healthcare workers (https://github.com/UCL/TLOmodel/tree/master/resources/healthsystem/human_resources/actual)

• Data for the Lifestyle Module parameters (@andrew-phillips-1 developed this offline from the DHS data; the DHS data comes in the same format for all countries; and so it should be possible to create scripts that estimate these parameters for any DHS dataset)

• Healthcare seeking: @winnga developed analyses using (DATA) to estimate the patterns of healthcare seeking behaviour, which are here: https://github.com/UCL/TLOmodel/blob/master/resources/ResourceFile_HealthSeekingBehaviour.csv (The scripts were developed offline but the method are described in his publications, linked here)

• Consumables availability data (@sakshimohan has developed scripts that produce estimates of Consumables definitions and their availability (https://github.com/UCL/TLOmodel/tree/master/resources/healthsystem/consumables): https://github.com/UCL/TLOmodel/tree/master/src/scripts/data_file_processing/healthsystem/consumables

• Data from the DHIS2 for calibration (@BinglingICL has developed a procedure of downloading the relevant datasets and processing them into the format required for calibration to model outputs. Those scripts are written in python but currently stored in a separate repo. The datafile themselves are stored offline but would need to be move into our shared OneDriver folder (/Data)

N.B. This leaves aside:

• The epidemiological parameters contained within other ResourceFiles, for which other processes and guidance we need to developed, and which will also benefit from labelling/categorising as per Label free parameters in modules #1036
• Equipment catalogue (developed by @sakshimohan), costing inputs (developed by @sakshimohan) and Bed Days inputs (developed by @tbhallett) that have tended to use bespoke data sources, and may not immediately be translatable into this format)

[BinglingICL]

Thanks very much Tim for creating this very clear issue path. Will complete the tasks in due course.

[tbhallett]

Thanks @BinglingICL

I believe the immediate next steps would be:

• @jkumwenda to familiarise himself with the first two of these data sources
• @tamuri to advise of the format that these scripts should be in (same/different repo / using third-party packages etc)
• @jkumwenda to make those changes in respect of the first two of these data sources
• @jkumwenda to develop process by which each of these types of data can be adapted to the same procedure, drawing on input from those who developed these files originally as/when needed.

[BinglingICL]

[ ] Data from the DHIS2 for calibration (@BinglingICL has developed a procedure of downloading the relevant datasets and processing them into the format required for calibration to model outputs. Those scripts are written in python but currently stored in a separate repo. The datafile themselves are stored offline but would need to be move into our shared OneDriver folder (/Data)

The data and scripts are now stored here: https://imperiallondon.sharepoint.com/:f:/r/sites/tlomodel-WP/Shared%20Documents/07%20-%20Data/DHIS2/Analysing%20Malawi%20Healthcare%20Service%20Usage%20-%20onedrive?csf=1&web=1&e=VJihfR. Once the standards are set, will reorganise the data and scripts.