Inclusion of Nonrandomized Studies of Interventions in Systematic Reviews of Interventions: Updated Guidance from the Agency for Health Care Research and Quality Effective Health Care Program

Objectives: We developed guidance to inform decisions regarding the inclusion of nonrandomized studies of interventions (NRSIs) in systematic reviews (SRs) of the effects of interventions.

Study Design And Setting: The guidance workgroup comprised SR experts and used an informal consensus generation method.

Results: Instead of recommending NRSI inclusion only if randomized controlled trials (RCTs) are insufficient to address the SR key question, different topics may require different decisions regarding NRSI inclusion. We identified important considerations to inform such decisions from topic refinement through protocol development. During topic scoping and refinement, considerations were related to the clinical decisional dilemma, adequacy of RCTs to address the key questions, risk of bias in NRSIs, and the extent to which NRSIs are likely to complement RCTs. When NRSIs are included, during SR team formation, familiarity with topic-specific data sources and advanced analytic methods for NRSIs should be considered. During protocol development, the decision regarding NRSI inclusion or exclusion should be justified, and potential implications explained. When NRSIs are included, the protocol should describe the processes for synthesizing evidence from RCTs and NRSIs and determining the overall strength of evidence.

Conclusion: We identified specific considerations for decisions regarding NRSI inclusion in SRs and highlight the importance of flexibility and transparency.