Development of Novel Composite Data Quality Scores to Evaluate Facility-level Data Quality in Electronic Data in Kenya: a Nationwide Retrospective Cohort Study

Overview

Journal BMC Health Serv Res

Publisher Biomed Central

Specialty Health Services

Date 2023 Oct 23

PMID 37872540

Authors

Beryne M Odeny

Anne Njoroge

Steve Gloyd

James P Hughes

Bradley H Wagenaar

Jacob Odhiambo

Lilly M Nyagah

Ayub Manya

Ooga Wesley Oghera

Nancy Puttkammer

Affiliations

Soon will be listed here.

Abstract

Background: In this evaluation, we aim to strengthen Routine Health Information Systems (RHIS) through the digitization of data quality assessment (DQA) processes. We leverage electronic data from the Kenya Health Information System (KHIS) which is based on the District Health Information System version 2 (DHIS2) to perform DQAs at scale. We provide a systematic guide to developing composite data quality scores and use these scores to assess data quality in Kenya.

Methods: We evaluated 187 HIV care facilities with electronic medical records across Kenya. Using quarterly, longitudinal KHIS data from January 2011 to June 2018 (total N = 30 quarters), we extracted indicators encompassing general HIV services including services to prevent mother-to-child transmission (PMTCT). We assessed the accuracy (the extent to which data were correct and free of error) of these data using three data-driven composite scores: 1) completeness score; 2) consistency score; and 3) discrepancy score. Completeness refers to the presence of the appropriate amount of data. Consistency refers to uniformity of data across multiple indicators. Discrepancy (measured on a Z-scale) refers to the degree of alignment (or lack thereof) of data with rules that defined the possible valid values for the data.

Results: A total of 5,610 unique facility-quarters were extracted from KHIS. The mean completeness score was 61.1% [standard deviation (SD) = 27%]. The mean consistency score was 80% (SD = 16.4%). The mean discrepancy score was 0.07 (SD = 0.22). A strong and positive correlation was identified between the consistency score and discrepancy score (correlation coefficient = 0.77), whereas the correlation of either score with the completeness score was low with a correlation coefficient of -0.12 (with consistency score) and -0.36 (with discrepancy score). General HIV indicators were more complete, but less consistent, and less plausible than PMTCT indicators.

Conclusion: We observed a lack of correlation between the completeness score and the other two scores. As such, for a holistic DQA, completeness assessment should be paired with the measurement of either consistency or discrepancy to reflect distinct dimensions of data quality. Given the complexity of the discrepancy score, we recommend the simpler consistency score, since they were highly correlated. Routine use of composite scores on KHIS data could enhance efficiencies in DQA at scale as digitization of health information expands and could be applied to other health sectors beyondHIV clinics.

References

Castelnuovo B, Kiragga A, Afayo V, Ncube M, Orama R, Magero S . Implementation of provider-based electronic medical records and improvement of the quality of data in a large HIV program in Sub-Saharan Africa. PLoS One. 2013; 7(12):e51631. PMC: 3524185. DOI: 10.1371/journal.pone.0051631. View

Jamieson L, Evans D, Berhanu R, Ismail N, Aucock S, Wallengren K . Data quality of drug-resistant tuberculosis and antiretroviral therapy electronic registers in South Africa. BMC Public Health. 2019; 19(1):1638. PMC: 6896407. DOI: 10.1186/s12889-019-7965-9. View

Kahn M, Raebel M, Glanz J, Riedlinger K, Steiner J . A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012; 50 Suppl:S21-9. PMC: 3833692. DOI: 10.1097/MLR.0b013e318257dd67. View

Gloyd S, Wagenaar B, Woelk G, Kalibala S . Opportunities and challenges in conducting secondary analysis of HIV programmes using data from routine health information systems and personal health information. J Int AIDS Soc. 2016; 19(5 Suppl 4):20847. PMC: 4956739. DOI: 10.7448/IAS.19.5.20847. View

Braa J, Heywood A, Sahay S . Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania. Bull World Health Organ. 2012; 90(5):379-84. PMC: 3341693. DOI: 10.2471/BLT.11.099580. View

Chaplin B, Meloni S, Eisen G, Jolayemi T, Banigbe B, Adeola J . Scale-up of networked HIV treatment in Nigeria: creation of an integrated electronic medical records system. Int J Med Inform. 2014; 84(1):58-68. DOI: 10.1016/j.ijmedinf.2014.09.006. View

Garrib A, Stoops N, Mckenzie A, Dlamini L, Govender T, Rohde J . An evaluation of the District Health Information System in rural South Africa. S Afr Med J. 2008; 98(7):549-52. View

Granato S, Gloyd S, Robinson J, Dali S, Ahoba I, Aka D . Results from a rapid national assessment of services for the prevention of mother-to-child transmission of HIV in Côte d'Ivoire. J Int AIDS Soc. 2016; 19(5 Suppl 4):20838. PMC: 4956736. DOI: 10.7448/IAS.19.5.20838. View

Nicol E, Hanmer L . Routine Health Information Systems in South Africa--Opportunities for Improvement. Stud Health Technol Inform. 2015; 216:993. View

10.

. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2017; 390(10100):1151-1210. PMC: 5605883. DOI: 10.1016/S0140-6736(17)32152-9. View

11.

Dehnavieh R, Haghdoost A, Khosravi A, Hoseinabadi F, Rahimi H, Poursheikhali A . The District Health Information System (DHIS2): A literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Inf Manag. 2018; 48(2):62-75. DOI: 10.1177/1833358318777713. View

12.

Rowe A . Potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. Am J Trop Med Hyg. 2009; 80(6):971-9. View

13.

Suthar A, Khalifa A, Joos O, Manders E, Abdul-Quader A, Amoyaw F . National health information systems for achieving the Sustainable Development Goals. BMJ Open. 2019; 9(5):e027689. PMC: 6530305. DOI: 10.1136/bmjopen-2018-027689. View

14.

OHagan R, Marx M, Finnegan K, Naphini P, Ngambi K, Laija K . National Assessment of Data Quality and Associated Systems-Level Factors in Malawi. Glob Health Sci Pract. 2017; 5(3):367-381. PMC: 5620335. DOI: 10.9745/GHSP-D-17-00177. View

15.

Chen H, Hailey D, Wang N, Yu P . A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014; 11(5):5170-207. PMC: 4053886. DOI: 10.3390/ijerph110505170. View

16.

Lee K, Weiskopf N, Pathak J . A Framework for Data Quality Assessment in Clinical Research Datasets. AMIA Annu Symp Proc. 2018; 2017:1080-1089. PMC: 5977591. View

17.

Nicol E, Dudley L, Bradshaw D . Assessing the quality of routine data for the prevention of mother-to-child transmission of HIV: An analytical observational study in two health districts with high HIV prevalence in South Africa. Int J Med Inform. 2016; 95:60-70. DOI: 10.1016/j.ijmedinf.2016.09.006. View

18.

Marchant T, Schellenberg J, Peterson S, Manzi F, Waiswa P, Hanson C . The use of continuous surveys to generate and continuously report high quality timely maternal and newborn health data at the district level in Tanzania and Uganda. Implement Sci. 2014; 9:112. PMC: 4160540. DOI: 10.1186/s13012-014-0112-1. View

19.

Ndira S, Rosenberger K, Wetter T . Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): a qualitative and quantitative comparative study. Methods Inf Med. 2008; 47(6):489-98. View

20.

Cohen J, Coleman E, Kangethe M . An importance-performance analysis of hospital information system attributes: A nurses' perspective. Int J Med Inform. 2015; 86:82-90. DOI: 10.1016/j.ijmedinf.2015.10.010. View