Case Studies to Explore the Optimal Use of Randomized and Nonrandomized Studies in Evidence Syntheses That Use GRADE

Objectives: Randomized controlled trials (RCTs) are the preferred source of evidence for the relative effect of healthcare interventions summarized in knowledge syntheses. Nonrandomized studies of interventions (NRSI) may provide replacement, sequential, or complementary evidence to RCTs. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach can provide different options for properly using RCTs and NRSI integrated in health syntheses. In this article, we discuss different implications on the certainty of evidence when authors consider the use of NRSI and RCTs in systematic reviews using GRADE. Although this is a GRADE-related article, it is not an official GRADE guidance or concept article.

Study Design And Setting: We present case studies used during GRADE working group meetings for discussion of the effects of using NRSI and RCTs on GRADE domains and on the certainty of evidence. Several concepts were discussed through iterative feedback with experts in GRADE methods and Cochrane authors. We compared suggested solutions for possible scenarios that can be met in evidence syntheses informing decisions and future guidance.

Results: Different scenarios for the use of RCTs and NRSI in evidence syntheses are presented, focusing on how different GRADE ratings between RCTs and NRSI affect the overall assessment of the evidence and possible health recommendations.

Conclusions: Considering differences and similarities grounded in the GRADE approach between NRSI and RCTs may help complement one another and maximize the value of knowledge syntheses and health recommendations.