An Algorithm Was Developed to Assign GRADE Levels of Evidence to Comparisons Within Systematic Reviews

Overview

Journal J Clin Epidemiol

Publisher Elsevier

Specialty Public Health

Date 2015 Sep 6

PMID 26341023

Citations 70

Authors

Alex Pollock

Sybil E Farmer

Marian C Brady

Peter Langhorne

Gillian E Mead

Jan Mehrholz

Frederike van Wijck

Philip J Wiffen

Affiliations

Soon will be listed here.

Abstract

Objectives: One recommended use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is supporting quality assessment of evidence of comparisons included within a Cochrane overview of reviews. Within our overview, reviewers found that current GRADE guidance was insufficient to make reliable and consistent judgments. To support our ratings, we developed an algorithm to grade quality of evidence using concrete rules.

Methods: Using a pragmatic, exploratory approach, we explored the challenges of applying GRADE levels of evidence and developed an algorithm to applying GRADE levels of evidence in a consistent and transparent approach. Our methods involved application of algorithms and formulas to samples of reviews, expert panel discussion, and iterative refinement and revision.

Results: The developed algorithm incorporated four key criteria: number of participants, risk of bias of trials, heterogeneity, and methodological quality of the review. A formula for applying GRADE level of evidence from the number of downgrades assigned by the algorithm was agreed.

Conclusion: Our algorithm which assigns GRADE levels of evidence using a set of concrete rules was successfully applied within our Cochrane overview. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.

Citing Articles

Protocol for a meta-review of interventions to prevent and manage ICU delirium.

Jones K, Kundakci B, Booth A, Pufulete M, Gibbison B BMJ Open. 2025; 15(2):e090815.

PMID: 39933812 PMC: 11815468. DOI: 10.1136/bmjopen-2024-090815.

): .

Gosling C, Cortese S, Radua J, Moher D, Delorme R, Solmi M BMJ Ment Health. 2024; 27(1.

PMID: 39694668 PMC: 11683946. DOI: 10.1136/bmjment-2024-301310.

A Systematic Review and Narrative Synthesis of Factors Affecting Pre-exposure Prophylaxis Willingness Among Black Women for HIV Prevention.

Sims Haynes A, Markham C, Schick V, Suchting R, Parthasarathy N, Choudhury S AIDS Behav. 2024; 29(1):101-132.

PMID: 39340583 PMC: 11739211. DOI: 10.1007/s10461-024-04491-z.

Evening dosing versus morning dosing of antihypertensive medications for nocturnal hypertension: a systematic review and meta-analysis of 107 randomized controlled trials.

Lee E, Wang S, Ng W, Ramdzan S, Tse E, Chan L J Hypertens. 2024; 42(10):1653-1664.

PMID: 39196688 PMC: 11356681. DOI: 10.1097/HJH.0000000000003783.

Non-pharmacological and non-surgical interventions to manage patients with knee osteoarthritis: An umbrella review 5-year update.

Ferreira R, Martins P, Goncalves R Osteoarthr Cartil Open. 2024; 6(3):100497.

PMID: 39040626 PMC: 11261791. DOI: 10.1016/j.ocarto.2024.100497.

References

Balshem H, Helfand M, Schunemann H, Oxman A, Kunz R, Brozek J . GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011; 64(4):401-6. DOI: 10.1016/j.jclinepi.2010.07.015. View

Dechartres A, Altman D, Trinquart L, Boutron I, Ravaud P . Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA. 2014; 312(6):623-30. DOI: 10.1001/jama.2014.8166. View

Moher D, Liberati A, Tetzlaff J, Altman D . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7):e1000097. PMC: 2707599. DOI: 10.1371/journal.pmed.1000097. View

Pollock A, Farmer S, Brady M, Langhorne P, Mead G, Mehrholz J . Interventions for improving upper limb function after stroke. Cochrane Database Syst Rev. 2014; (11):CD010820. PMC: 6469541. DOI: 10.1002/14651858.CD010820.pub2. View

Guyatt G, Oxman A, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336(7650):924-6. PMC: 2335261. DOI: 10.1136/bmj.39489.470347.AD. View

Guyatt G, Oxman A, Schunemann H, Tugwell P, Knottnerus A . GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2010; 64(4):380-2. DOI: 10.1016/j.jclinepi.2010.09.011. View

Savovic J, Jones H, Altman D, Harris R, Juni P, Pildal J . Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012; 157(6):429-38. DOI: 10.7326/0003-4819-157-6-201209180-00537. View

Dechartres A, Trinquart L, Boutron I, Ravaud P . Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ. 2013; 346:f2304. PMC: 3634626. DOI: 10.1136/bmj.f2304. View

Shea B, Grimshaw J, Wells G, Boers M, Andersson N, Hamel C . Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007; 7:10. PMC: 1810543. DOI: 10.1186/1471-2288-7-10. View

10.

Straus S, Shepperd S . Challenges in guideline methodology. J Clin Epidemiol. 2011; 64(4):347-8. DOI: 10.1016/j.jclinepi.2011.01.003. View

11.

Berkman N, Lohr K, Morgan L, Kuo T, Morton S . Interrater reliability of grading strength of evidence varies with the complexity of the evidence in systematic reviews. J Clin Epidemiol. 2013; 66(10):1105-1117.e1. DOI: 10.1016/j.jclinepi.2013.06.002. View

12.

Guyatt G, Oxman A, Akl E, Kunz R, Vist G, Brozek J . GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011; 64(4):383-94. DOI: 10.1016/j.jclinepi.2010.04.026. View

13.

Shea B, Hamel C, Wells G, Bouter L, Kristjansson E, Grimshaw J . AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009; 62(10):1013-20. DOI: 10.1016/j.jclinepi.2008.10.009. View

14.

Guyatt G, Oxman A, Vist G, Kunz R, Brozek J, Alonso-Coello P . GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol. 2011; 64(4):407-15. DOI: 10.1016/j.jclinepi.2010.07.017. View