Validating the QCOVID Risk Prediction Algorithm for Risk of Mortality from COVID-19 in the Adult Population in Wales, UK

Overview

Journal Int J Popul Data Sci

Specialty Public Health

Date 2022 Mar 21

PMID 35310465

Authors

Jane Lyons

Vahe Nafilyan

Ashley Akbari

Gareth Davies

Rowena Griffiths

Ewen M Harrison

Julia Hippisley-Cox

Joe Hollinghurst

Kamlesh Khunti

Laura North

Aziz Sheikh

Fatemeh Torabi

Ronan A Lyons

Affiliations

Soon will be listed here.

Abstract

Introduction: COVID-19 risk prediction algorithms can be used to identify at-risk individuals from short-term serious adverse COVID-19 outcomes such as hospitalisation and death. It is important to validate these algorithms in different and diverse populations to help guide risk management decisions and target vaccination and treatment programs to the most vulnerable individuals in society.

Objectives: To validate externally the QCOVID risk prediction algorithm that predicts mortality outcomes from COVID-19 in the adult population of Wales, UK.

Methods: We conducted a retrospective cohort study using routinely collected individual-level data held in the Secure Anonymised Information Linkage (SAIL) Databank. The cohort included individuals aged between 19 and 100 years, living in Wales on 24 January 2020, registered with a SAIL-providing general practice, and followed-up to death or study end (28 July 2020). Demographic, primary and secondary healthcare, and dispensing data were used to derive all the predictor variables used to develop the published QCOVID algorithm. Mortality data were used to define time to confirmed or suspected COVID-19 death. Performance metrics, including R values (explained variation), Brier scores, and measures of discrimination and calibration were calculated for two periods (24 January-30 April 2020 and 1 May-28 July 2020) to assess algorithm performance.

Results: 1,956,760 individuals were included. 1,192 (0.06%) and 610 (0.03%) COVID-19 deaths occurred in the first and second time periods, respectively. The algorithms fitted the Welsh data and population well, explaining 68.8% (95% CI: 66.9-70.4) of the variation in time to death, Harrell's C statistic: 0.929 (95% CI: 0.921-0.937) and D statistic: 3.036 (95% CI: 2.913-3.159) for males in the first period. Similar results were found for females and in the second time period for both sexes.

Conclusions: The QCOVID algorithm developed in England can be used for public health risk management for the adult Welsh population.

Citing Articles

Development and validation of a risk prediction model for hospital admission in COVID-19 patients presenting to primary care.

Wynants L, Broers N, Platteel T, Venekamp R, Barten D, Leers M Eur J Gen Pract. 2024; 30(1):2339488.

PMID: 38682305 PMC: 11060008. DOI: 10.1080/13814788.2024.2339488.

Cardiac interventions in Wales: A comparison of benefits between NHS Wales specialties.

Davies G, Akbari A, Bailey R, Evans L, Smith K, Goodfellow J PLoS One. 2024; 19(2):e0297049.

PMID: 38335178 PMC: 10857708. DOI: 10.1371/journal.pone.0297049.

External validation of the QCovid 2 and 3 risk prediction algorithms for risk of COVID-19 hospitalisation and mortality in adults: a national cohort study in Scotland.

Kerr S, Millington T, Rudan I, McCowan C, Tibble H, Jeffrey K BMJ Open. 2023; 13(12):e075958.

PMID: 38151278 PMC: 10753764. DOI: 10.1136/bmjopen-2023-075958.

Vaccine effectiveness for prevention of covid-19 related hospital admission during pregnancy in England during the alpha and delta variant dominant periods of the SARS-CoV-2 pandemic: population based cohort study.

Bosworth M, Schofield R, Ayoubkhani D, Charlton L, Nafilyan V, Khunti K BMJ Med. 2023; 2(1):e000403.

PMID: 37564827 PMC: 10410807. DOI: 10.1136/bmjmed-2022-000403.

Rationale for the shielding policy for clinically vulnerable people in the UK during the COVID-19 pandemic: a qualitative study.

Porter A, Akbari A, Carson-Stevens A, Dale J, Dixon L, Edwards A BMJ Open. 2023; 13(8):e073464.

PMID: 37541747 PMC: 10407356. DOI: 10.1136/bmjopen-2023-073464.

References

Royston P, Sauerbrei W . A new measure of prognostic separation in survival data. Stat Med. 2004; 23(5):723-48. DOI: 10.1002/sim.1621. View

Singh A, Gillies C, Singh R, Singh A, Chudasama Y, Coles B . Prevalence of co-morbidities and their association with mortality in patients with COVID-19: A systematic review and meta-analysis. Diabetes Obes Metab. 2020; 22(10):1915-1924. PMC: 7361304. DOI: 10.1111/dom.14124. View

Sattar N, McInnes I, McMurray J . Obesity Is a Risk Factor for Severe COVID-19 Infection: Multiple Potential Mechanisms. Circulation. 2020; 142(1):4-6. DOI: 10.1161/CIRCULATIONAHA.120.047659. View

Lyons R, Jones K, John G, Brooks C, Verplancke J, Ford D . The SAIL databank: linking multiple health and social care datasets. BMC Med Inform Decis Mak. 2009; 9:3. PMC: 2648953. DOI: 10.1186/1472-6947-9-3. View

Harrell Jr F, Lee K, Mark D . Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15(4):361-87. DOI: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. View

Ford D, Jones K, Verplancke J, Lyons R, John G, Brown G . The SAIL Databank: building a national architecture for e-health research and evaluation. BMC Health Serv Res. 2009; 9:157. PMC: 2744675. DOI: 10.1186/1472-6963-9-157. View

Wynants L, Van Calster B, Collins G, Riley R, Heinze G, Schuit E . Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020; 369:m1328. PMC: 7222643. DOI: 10.1136/bmj.m1328. View

Lyons J, Akbari A, Torabi F, Davies G, North L, Griffiths R . Understanding and responding to COVID-19 in Wales: protocol for a privacy-protecting data platform for enhanced epidemiology and evaluation of interventions. BMJ Open. 2020; 10(10):e043010. PMC: 7580065. DOI: 10.1136/bmjopen-2020-043010. View

Collins G, Reitsma J, Altman D, Moons K . Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015; 162(1):55-63. DOI: 10.7326/M14-0697. View

10.

Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y . Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020; 395(10223):497-506. PMC: 7159299. DOI: 10.1016/S0140-6736(20)30183-5. View

11.

Hollinghurst J, Lyons J, Fry R, Akbari A, Gravenor M, Watkins A . The impact of COVID-19 on adjusted mortality risk in care homes for older adults in Wales, UK: a retrospective population-based cohort study for mortality in 2016-2020. Age Ageing. 2020; 50(1):25-31. PMC: 7546151. DOI: 10.1093/ageing/afaa207. View

12.

Docherty A, Harrison E, Green C, Hardwick H, Pius R, Norman L . Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ. 2020; 369:m1985. PMC: 7243036. DOI: 10.1136/bmj.m1985. View

13.

Richardson S, Hirsch J, Narasimhan M, Crawford J, McGinn T, Davidson K . Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area. JAMA. 2020; 323(20):2052-2059. PMC: 7177629. DOI: 10.1001/jama.2020.6775. View

14.

Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z . Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020; 395(10229):1054-1062. PMC: 7270627. DOI: 10.1016/S0140-6736(20)30566-3. View

15.

Austin P, Steyerberg E, Putter H . Fine-Gray subdistribution hazard models to simultaneously estimate the absolute risk of different event types: Cumulative total failure probability may exceed 1. Stat Med. 2021; 40(19):4200-4212. PMC: 8360146. DOI: 10.1002/sim.9023. View

16.

Benchimol E, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I . The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015; 12(10):e1001885. PMC: 4595218. DOI: 10.1371/journal.pmed.1001885. View

17.

Harrison S, Fazio-Eynullayeva E, Lane D, Underhill P, Lip G . Comorbidities associated with mortality in 31,461 adults with COVID-19 in the United States: A federated electronic medical record analysis. PLoS Med. 2020; 17(9):e1003321. PMC: 7482833. DOI: 10.1371/journal.pmed.1003321. View

18.

Nafilyan V, Humberstone B, Mehta N, Diamond I, Coupland C, Lorenzi L . An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: a national validation cohort study in England. Lancet Digit Health. 2021; 3(7):e425-e433. PMC: 8148652. DOI: 10.1016/S2589-7500(21)00080-7. View

19.

Davey Smith G, Spiegelhalter D . Shielding from covid-19 should be stratified by risk. BMJ. 2020; 369:m2063. DOI: 10.1136/bmj.m2063. View

20.

Clift A, Coupland C, Keogh R, Diaz-Ordaz K, Williamson E, Harrison E . Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study. BMJ. 2020; 371:m3731. PMC: 7574532. DOI: 10.1136/bmj.m3731. View