Validation of Prognostic Models Predicting Mortality or ICU Admission in Patients with COVID-19 in Low- and Middle-income Countries: a Global Individual Participant Data Meta-analysis

Overview

Journal Diagn Progn Res

Publisher Biomed Central

Date 2024 Dec 19

PMID 39696542

Authors

Johanna A A Damen

Banafsheh Arshi

Maarten van Smeden

Silvia Bertagnolio

Janet V Diaz

Ronaldo Silva

Soe Soe Thwin

Laure Wynants

Karel G M Moons

Affiliations

Soon will be listed here.

Abstract

Background: We evaluated the performance of prognostic models for predicting mortality or ICU admission in hospitalized patients with COVID-19 in the World Health Organization (WHO) Global Clinical Platform, a repository of individual-level clinical data of patients hospitalized with COVID-19, including in low- and middle-income countries (LMICs).

Methods: We identified eligible multivariable prognostic models for predicting overall mortality and ICU admission during hospital stay in patients with confirmed or suspected COVID-19 from a living review of COVID-19 prediction models. These models were evaluated using data contributed to the WHO Global Clinical Platform for COVID-19 from nine LMICs (Burkina Faso, Cameroon, Democratic Republic of Congo, Guinea, India, Niger, Nigeria, Zambia, and Zimbabwe). Model performance was assessed in terms of discrimination and calibration.

Results: Out of 144 eligible models, 140 were excluded due to a high risk of bias, predictors unavailable in LIMCs, or insufficient model description. Among 11,338 participants, the remaining models showed good discrimination for predicting in-hospital mortality (3 models), with areas under the curve (AUCs) ranging between 0.76 (95% CI 0.71-0.81) and 0.84 (95% CI 0.77-0.89). An AUC of 0.74 (95% CI 0.70-0.78) was found for predicting ICU admission risk (one model). All models showed signs of miscalibration and overfitting, with extensive heterogeneity between countries.

Conclusions: Among the available COVID-19 prognostic models, only a few could be validated on data collected from LMICs, mainly due to limited predictor availability. Despite their discriminative ability, selected models for mortality prediction or ICU admission showed varying and suboptimal calibration.

References

Wolff R, Moons K, Riley R, Whiting P, Westwood M, Collins G . PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Ann Intern Med. 2019; 170(1):51-58. DOI: 10.7326/M18-1376. View

Moons K, Wolff R, Riley R, Whiting P, Westwood M, Collins G . PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. Ann Intern Med. 2019; 170(1):W1-W33. DOI: 10.7326/M18-1377. View

Moons K, Altman D, Reitsma J, Ioannidis J, Macaskill P, Steyerberg E . Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015; 162(1):W1-73. DOI: 10.7326/M14-0698. View

van Klaveren D, Zanos T, Nelson J, Levy T, Park J, Retel Helmrich I . Prognostic models for COVID-19 needed updating to warrant transportability over time and space. BMC Med. 2022; 20(1):456. PMC: 9686462. DOI: 10.1186/s12916-022-02651-3. View

Gupta R, Marks M, Samuels T, Luintel A, Rampling T, Chowdhury H . Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. Eur Respir J. 2020; 56(6). PMC: 7518075. DOI: 10.1183/13993003.03498-2020. View

de Jong V, Rousset R, Antonio-Villa N, Buenen A, Van Calster B, Bello-Chavolla O . Clinical prediction models for mortality in patients with covid-19: external validation and individual participant data meta-analysis. BMJ. 2022; 378:e069881. PMC: 9273913. DOI: 10.1136/bmj-2021-069881. View

Gupta M, Wahl B, Adhikari B, Bar-Zeev N, Bhandari S, Coria A . The need for COVID-19 research in low- and middle-income countries. Glob Health Res Policy. 2020; 5:33. PMC: 7326528. DOI: 10.1186/s41256-020-00159-y. View

Debray T, Damen J, Snell K, Ensor J, Hooft L, Reitsma J . A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017; 356:i6460. DOI: 10.1136/bmj.i6460. View

Telenti A, Arvin A, Corey L, Corti D, Diamond M, Garcia-Sastre A . After the pandemic: perspectives on the future trajectory of COVID-19. Nature. 2021; 596(7873):495-504. DOI: 10.1038/s41586-021-03792-w. View

10.

Wang K, Zuo P, Liu Y, Zhang M, Zhao X, Xie S . Clinical and Laboratory Predictors of In-hospital Mortality in Patients With Coronavirus Disease-2019: A Cohort Study in Wuhan, China. Clin Infect Dis. 2020; 71(16):2079-2088. PMC: 7197616. DOI: 10.1093/cid/ciaa538. View

11.

Schultz M, Gebremariam T, Park C, Pisani L, Sivakorn C, Taran S . Pragmatic Recommendations for the Use of Diagnostic Testing and Prognostic Models in Hospitalized Patients with Severe COVID-19 in Low- and Middle-Income Countries. Am J Trop Med Hyg. 2021; 104(3_Suppl):34-47. PMC: 7957242. DOI: 10.4269/ajtmh.20-0730. View

12.

Bennouar S, Bachir Cherif A, Kessira A, Bennouar D, Abdi S . Development and validation of a laboratory risk score for the early prediction of COVID-19 severity and in-hospital mortality. Intensive Crit Care Nurs. 2021; 64:103012. PMC: 7834685. DOI: 10.1016/j.iccn.2021.103012. View

13.

Levin A, Owusu-Boaitey N, Pugh S, Fosdick B, Zwi A, Malani A . Assessing the burden of COVID-19 in developing countries: systematic review, meta-analysis and public policy implications. BMJ Glob Health. 2022; 7(5). PMC: 9136695. DOI: 10.1136/bmjgh-2022-008477. View

14.

Marshall A, Altman D, Holder R, Royston P . Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009; 9:57. PMC: 2727536. DOI: 10.1186/1471-2288-9-57. View

15.

Riley R, Ensor J, Snell K, Debray T, Altman D, Moons K . External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ. 2016; 353:i3140. PMC: 4916924. DOI: 10.1136/bmj.i3140. View

16.

Chandna A, Mahajan R, Gautam P, Mwandigha L, Gunasekaran K, Bhusan D . Facilitating Safe Discharge Through Predicting Disease Progression in Moderate Coronavirus Disease 2019 (COVID-19): A Prospective Cohort Study to Develop and Validate a Clinical Prediction Model in Resource-Limited Settings. Clin Infect Dis. 2022; 75(1):e368-e379. PMC: 9129107. DOI: 10.1093/cid/ciac224. View

17.

Zhou Y, He Y, Yang H, Yu H, Wang T, Chen Z . Development and validation a nomogram for predicting the risk of severe COVID-19: A multi-center study in Sichuan, China. PLoS One. 2020; 15(5):e0233328. PMC: 7233581. DOI: 10.1371/journal.pone.0233328. View

18.

Wynants L, Van Calster B, Collins G, Riley R, Heinze G, Schuit E . Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020; 369:m1328. PMC: 7222643. DOI: 10.1136/bmj.m1328. View

19.

Khedr A, Hennawi H, Rauf I, Khan M, Mushtaq H, Lodhi H . Differential mortality with COVID-19 and invasive mechanical ventilation between high-income and low-and middle-income countries: a systematic review, meta-analysis, and meta-regression. Infez Med. 2022; 30(1):51-58. PMC: 8929741. DOI: 10.53854/liim-3001-6. View

20.

Collins G, Reitsma J, Altman D, Moons K . Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015; 162(1):55-63. DOI: 10.7326/M14-0697. View