Challenges in Applying the GRADE Approach in Public Health Guidelines and Systematic Reviews: a Concept Article from the GRADE Public Health Group

Overview

Journal J Clin Epidemiol

Publisher Elsevier

Specialty Public Health

Date 2021 Jan 21

PMID 33476768

Citations 24

Authors

Michele Hilton Boon

Hilary Thomson

Beth Shaw

Elie A Akl

Stefan K Lhachimi

Jesus Lopez-Alcalde

Miloslav Klugar

Leslie Choi

Zuleika Saz-Parkinson

Reem A Mustafa

Miranda W Langendam

Olivia Crane

Rebecca L Morgan

Eva Rehfuess

Bradley C Johnston

Lee Yee Chong

Gordon H Guyatt

Holger J Schunemann

Srinivasa Vittal Katikireddi

Affiliations

Soon will be listed here.

Abstract

Background And Objective: This article explores the need for conceptual advances and practical guidance in the application of the GRADE approach within public health contexts.

Methods: We convened an expert workshop and conducted a scoping review to identify challenges experienced by GRADE users in public health contexts. We developed this concept article through thematic analysis and an iterative process of consultation and discussion conducted with members electronically and at three GRADE Working Group meetings.

Results: Five priority issues can pose challenges for public health guideline developers and systematic reviewers when applying GRADE: (1) incorporating the perspectives of diverse stakeholders; (2) selecting and prioritizing health and "nonhealth" outcomes; (3) interpreting outcomes and identifying a threshold for decision-making; (4) assessing certainty of evidence from diverse sources, including nonrandomized studies; and (5) addressing implications for decision makers, including concerns about conditional recommendations. We illustrate these challenges with examples from public health guidelines and systematic reviews, identifying gaps where conceptual advances may facilitate the consistent application or further development of the methodology and provide solutions.

Conclusion: The GRADE Public Health Group will respond to these challenges with solutions that are coherent with existing guidance and can be consistently implemented across public health decision-making contexts.

Citing Articles

CYP2D6 genotype and associated 5-HT receptor antagonist outcomes: A systematic review and meta-analysis.

Moore C, Williams E, Dyas R, Halman A, Stenta T, Khatri D Clin Transl Sci. 2025; 18(2):e70108.

PMID: 39899439 PMC: 11789932. DOI: 10.1111/cts.70108.

Nutrition users' guides: systematic reviews part 1 -structured guide for methodological assessment, interpretation and application of systematic reviews and meta-analyses of non-randomised nutritional epidemiology studies.

Zeraatkar D, de Souza R, Guyatt G, Bala M, Alonso-Coello P, Johnston B BMJ Nutr Prev Health. 2025; 7(2):e000835.

PMID: 39882294 PMC: 11773667. DOI: 10.1136/bmjnph-2023-000835.

Population dynamics and digitalization: implications for COVID-19 data sources in South Africa-a scoping review.

Akokuwebe M, Palamuleni M, Idemudia E Front Public Health. 2025; 12:1537057.

PMID: 39882125 PMC: 11779501. DOI: 10.3389/fpubh.2024.1537057.

Integrating Ethics in Public Health Guideline Development: Case Study of Guidelines on Respiratory Isolation for Persons With Tuberculosis in Community Settings.

Kates O, Shah M J Infect Dis. 2024; 231(1):23-30.

PMID: 39373254 PMC: 11793056. DOI: 10.1093/infdis/jiae478.

Should my child be given antibiotics? A systematic review of parental decision making in rural and remote locations.

Marsh S, Parsafar S, Byrne M Antimicrob Resist Infect Control. 2024; 13(1):105.

PMID: 39294826 PMC: 11412025. DOI: 10.1186/s13756-024-01409-1.

References

Threlfall A, Meah S, Fischer A, Cookson R, Rutter H, Kelly M . The appraisal of public health interventions: the use of theory. J Public Health (Oxf). 2014; 37(1):166-71. DOI: 10.1093/pubmed/fdu044. View

von Philipsborn P, Stratil J, Burns J, Busert L, Pfadenhauer L, Polus S . Environmental interventions to reduce the consumption of sugar-sweetened beverages and their effects on health. Cochrane Database Syst Rev. 2019; 6:CD012292. PMC: 6564085. DOI: 10.1002/14651858.CD012292.pub2. View

Murad M, Mustafa R, Schunemann H, Sultan S, Santesso N . Rating the certainty in evidence in the absence of a single estimate of effect. Evid Based Med. 2017; 22(3):85-87. PMC: 5502230. DOI: 10.1136/ebmed-2017-110668. View

Pryce J, Richardson M, Lengeler C . Insecticide-treated nets for preventing malaria. Cochrane Database Syst Rev. 2018; 11:CD000363. PMC: 6418392. DOI: 10.1002/14651858.CD000363.pub3. View

Choi L, Majambere S, Wilson A . Larviciding to prevent malaria transmission. Cochrane Database Syst Rev. 2019; 8:CD012736. PMC: 6699674. DOI: 10.1002/14651858.CD012736.pub2. View

Pega F, Liu S, Walter S, Lhachimi S . Unconditional cash transfers for assistance in humanitarian disasters: effect on use of health services and health outcomes in low- and middle-income countries. Cochrane Database Syst Rev. 2015; (9):CD011247. PMC: 9157652. DOI: 10.1002/14651858.CD011247.pub2. View

Pottie K, Welch V, Morton R, Akl E, Eslava-Schmalbach J, Katikireddi V . GRADE equity guidelines 4: considering health equity in GRADE guideline development: evidence to decision process. J Clin Epidemiol. 2017; 90:84-91. PMC: 6538528. DOI: 10.1016/j.jclinepi.2017.08.001. View

Griffiths S, Jewell T, Donnelly P . Public health in practice: the three domains of public health. Public Health. 2005; 119(10):907-13. PMC: 7111730. DOI: 10.1016/j.puhe.2005.01.010. View

Benach J, Malmusi D, Yasui Y, Martinez J . A new typology of policies to tackle health inequalities and scenarios of impact based on Rose's population approach. J Epidemiol Community Health. 2012; 67(3):286-91. DOI: 10.1136/jech-2011-200363. View

10.

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin C, Gulmezoglu M . Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015; 12(10):e1001895. PMC: 4624425. DOI: 10.1371/journal.pmed.1001895. View

11.

Burford B, Rehfuess E, Schunemann H, Akl E, Waters E, Armstrong R . Assessing evidence in public health: the added value of GRADE. J Public Health (Oxf). 2012; 34(4):631-5. DOI: 10.1093/pubmed/fds092. View

12.

Thayer K, Schunemann H . Using GRADE to respond to health questions with different levels of urgency. Environ Int. 2016; 92-93:585-9. DOI: 10.1016/j.envint.2016.03.027. View

13.

Rehfuess E, Akl E . Current experience with applying the GRADE approach to public health interventions: an empirical study. BMC Public Health. 2013; 13:9. PMC: 3546302. DOI: 10.1186/1471-2458-13-9. View

14.

Pega F, Liu S, Walter S, Pabayo R, Saith R, Lhachimi S . Unconditional cash transfers for reducing poverty and vulnerabilities: effect on use of health services and health outcomes in low- and middle-income countries. Cochrane Database Syst Rev. 2017; 11:CD011135. PMC: 6486161. DOI: 10.1002/14651858.CD011135.pub2. View

15.

Schunemann H, Cuello C, Akl E, Mustafa R, Meerpohl J, Thayer K . GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2018; 111:105-114. PMC: 6692166. DOI: 10.1016/j.jclinepi.2018.01.012. View

16.

Dedios M, Esperato A, De-Regil L, Pena-Rosas J, Norris S . Improving the adaptability of WHO evidence-informed guidelines for nutrition actions: results of a mixed methods evaluation. Implement Sci. 2017; 12(1):39. PMC: 5361729. DOI: 10.1186/s13012-017-0571-2. View

17.

Hultcrantz M, Rind D, Akl E, Treweek S, Mustafa R, Iorio A . The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017; 87:4-13. PMC: 6542664. DOI: 10.1016/j.jclinepi.2017.05.006. View

18.

Alonso-Coello P, Schunemann H, Moberg J, Brignardello-Petersen R, Akl E, Davoli M . GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016; 353:i2016. DOI: 10.1136/bmj.i2016. View

19.

Burns J, Boogaard H, Polus S, Pfadenhauer L, Rohwer A, van Erp A . Interventions to reduce ambient particulate matter air pollution and their effect on health. Cochrane Database Syst Rev. 2019; 5:CD010919. PMC: 6526394. DOI: 10.1002/14651858.CD010919.pub2. View

20.

Fischer A, Threlfall A, Meah S, Cookson R, Rutter H, Kelly M . The appraisal of public health interventions: an overview. J Public Health (Oxf). 2013; 35(4):488-94. DOI: 10.1093/pubmed/fdt076. View