Evaluating the Methodology of Studies Conducted During the Global COVID-19 Pandemic: A Systematic Review of Randomized Controlled Trials

Background: The therapeutic evidence collected from well-designed studies is needed to help manage the global pandemic of the coronavirus disease 2019 (COVID-19). Evaluating the quality of therapeutic data collected during this most recent pandemic is important for improving future clinical research under similar circumstances.

Objective: To assess the methodological quality and variability in implementation of randomized controlled trials (RCTs) for treating COVID-19, and to analyze the support that should be provided to improve data collected during an urgent pandemic situation.

Search Strategy: PubMed, Excerpta Medica Database, China National Knowledge Infrastructure, Wanfang, and Chongqing VIP, and the preprint repositories including Social Science Research Network and MedRxiv were systematically searched, up to September 30, 2020, using the keywords "coronavirus disease 2019 (COVID-19)," "2019 novel coronavirus (2019-nCoV)," "severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2)," "novel coronavirus pneumonia (NCP)," "randomized controlled trial (RCT)" and "random."

Inclusion Criteria: RCTs studying the treatment of COVID-19 were eligible for inclusion.

Data Extraction And Analysis: Screening of published RCTs for inclusion and data extraction were each conducted by two researchers. Analysis of general information on COVID-19 RCTs was done using descriptive statistics. Methodological quality was assessed using the risk-of-bias tools in the Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0). Variability in implementation was assessed by comparing consistency between RCT reports and registration information.

Results: A total of 5886 COVID-19 RCTs were identified. Eighty-one RCTs were finally included, of which, 45 had registration information. Methodological quality of the RTCs was not optimal due to deficiencies in five main domains: allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Comparisons of consistency between published protocols and registration information showed that the 45 RCTs with registration information had common deviations in seven items: inclusion and exclusion criteria, sample size, outcomes, research sites of recruitment, interventions, and blinding.

Conclusion: The methodological quality of COVID-19 RCTs conducted in early to mid 2020 was consistently low and variability in implementation was common. More support for implementing high-quality methodology is needed to obtain the quality of therapeutic evidence needed to provide positive guidance for clinical care. We make an urgent appeal for accelerating the construction of a collaborative sharing platform and preparing multidisciplinary talent and professional teams to conduct excellent clinical research when faced with epidemic diseases of the future. Further, variability in RCT implementation should be clearly reported and interpreted to improve the utility of data resulting from those trials.