Tools for Assessing the Content of Guidelines Are Needed to Enable Their Effective Use--a Systematic Comparison

Overview

Journal BMC Res Notes

Publisher Biomed Central

Specialties Biology
General Medicine

Date 2014 Nov 28

PMID 25427972

Citations 12

Authors

Michaela Eikermann

Nicole Holzmann

Ulrich Siering

Alric Ruther

Affiliations

Soon will be listed here.

Abstract

Background: To ensure that clinical practice guidelines (CPGs) form a sound basis for decision-making in health care, it is necessary to be able to reliably assess and ensure their quality. This results in the need to assess the content of guidelines systematically, particularly with regard to the validity of their recommendations.The aim of the present analysis was to determine the suitability and applicability of frequently used assessment tools for evidence syntheses with regard to the assessment of guideline content.

Methods: We conducted a systematic comparison and analysis of established tools for the assessment of evidence syntheses (guidelines, systematic reviews, health technology assessments). The tools analyzed were: ADAPTE, AGREE II, AMSTAR, GLIA and the INAHTA checklist. We analyzed methodological steps related to the assessment of the reliability and validity of guideline recommendations. Data were extracted and analyzed by two persons independently of one another.

Results: Widely used tools for the methodological assessment of evidence syntheses are not suitable for a comprehensive content-related assessment. They remain mostly at the level of assessment of the documentation of processes. Some tools assess selected content-related aspects, but operationalization is either unspecific or lacking.

Conclusion: None of the tools analyzed enables the structured and comprehensive assessment of the content of guideline recommendations with special regard to their reliability and validity. All tools contribute towards the judicious use of evidence syntheses by supporting their systematic development or assessment. However, further progress is needed, particularly with regard to the assessment of content quality. This includes comprehensive operationalization and documentation of the assessment process to ensure reliability and validity, and therefore to enable the effective use of trustworthy guidelines in the health care system.

Citing Articles

Clinical Practice Guidelines in an Era of Accountability, Saudi Arabia: A Call for Action.

Abdellatif H, Al-Muallem A, Almansoof A, AlRohaily S, Alzahrani A, AlGhodaier H J Epidemiol Glob Health. 2023; 13(3):391-396.

PMID: 37450240 PMC: 10469134. DOI: 10.1007/s44197-023-00135-y.

Treatments for non-small cell lung cancer: a systematic quality assessment of clinical practice guidelines.

Cortes-Jofre M, Madera M, Tirado-Amador L, Asenjo-Lobos C, Bonfill-Cosp X Clin Transl Oncol. 2023; 25(12):3541-3555.

PMID: 37254015 DOI: 10.1007/s12094-023-03223-4.

Health equity considerations in guideline development: a rapid scoping review.

Shaver N, Bennett A, Beck A, Skidmore B, Traversy G, Brouwers M CMAJ Open. 2023; 11(2):E357-E371.

PMID: 37171906 PMC: 10139082. DOI: 10.9778/cmajo.20220130.

Quality of clinical practice guidelines and recommendations for the management of pain, sedation, delirium and iatrogenic withdrawal in pediatric intensive care: a systematic review protocol.

MacDonald I, Perez M, Amiet V, Trombert A, Ramelet A BMJ Paediatr Open. 2022; 6(1).

PMID: 36053608 PMC: 8852722. DOI: 10.1136/bmjpo-2021-001293.

Representing narrative evidence as clinical evidence logic statements.

Lacson R, Eskian M, Cochon L, Gujrathi I, Licaros A, Zhao A JAMIA Open. 2022; 5(2):ooac024.

PMID: 35474718 PMC: 9030217. DOI: 10.1093/jamiaopen/ooac024.

References

McMurray J, Swedberg K . Treatment of chronic heart failure: a comparison between the major guidelines. Eur Heart J. 2006; 27(15):1773-7. DOI: 10.1093/eurheartj/ehl123. View

Burgers J, Bailey J, Klazinga N, van der Bij A, Grol R, Feder G . Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries. Diabetes Care. 2002; 25(11):1933-9. DOI: 10.2337/diacare.25.11.1933. View

Guyatt G, Oxman A, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008; 336(7650):924-6. PMC: 2335261. DOI: 10.1136/bmj.39489.470347.AD. View

Guyatt G, Oxman A, Kunz R, Jaeschke R, Helfand M, Liberati A . Incorporating considerations of resources use into grading recommendations. BMJ. 2008; 336(7654):1170-3. PMC: 2394579. DOI: 10.1136/bmj.39504.506319.80. View

Siering U, Eikermann M, Hausner E, Hoffmann-Esser W, Neugebauer E . Appraisal tools for clinical practice guidelines: a systematic review. PLoS One. 2013; 8(12):e82915. PMC: 3857289. DOI: 10.1371/journal.pone.0082915. View

Watine J, Friedberg B, Nagy E, Onody R, Oosterhuis W, Bunting P . Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners. Clin Chem. 2006; 52(1):65-72. DOI: 10.1373/clinchem.2005.056952. View

Eyding D, Lelgemann M, Grouven U, Harter M, Kromp M, Kaiser T . Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010; 341:c4737. PMC: 2954275. DOI: 10.1136/bmj.c4737. View

Burls A . AGREE II-improving the quality of clinical care. Lancet. 2010; 376(9747):1128-9. DOI: 10.1016/S0140-6736(10)61034-3. View

Fervers B, Burgers J, Voellinger R, Brouwers M, Browman G, Graham I . Guideline adaptation: an approach to enhance efficiency in guideline development and improve utilisation. BMJ Qual Saf. 2011; 20(3):228-36. DOI: 10.1136/bmjqs.2010.043257. View

10.

Matthys J, De Meyere M, Van Driel M, De Sutter A . Differences among international pharyngitis guidelines: not just academic. Ann Fam Med. 2007; 5(5):436-43. PMC: 2000301. DOI: 10.1370/afm.741. View

11.

Shiffman R, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G . The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005; 5:23. PMC: 1190181. DOI: 10.1186/1472-6947-5-23. View

12.

Hill K, Lalor E . How useful is an online tool to facilitate guideline implementation? Feasibility study of using eGLIA by stroke clinicians in Australia. Qual Saf Health Care. 2009; 18(2):157-9. DOI: 10.1136/qshc.2007.025635. View

13.

Guyatt G, Oxman A, Kunz R, Vist G, Falck-Ytter Y, Schunemann H . What is "quality of evidence" and why is it important to clinicians?. BMJ. 2008; 336(7651):995-8. PMC: 2364804. DOI: 10.1136/bmj.39490.551019.BE. View

14.

Kung J, Miller R, Mackowiak P . Failure of clinical practice guidelines to meet institute of medicine standards: Two more decades of little, if any, progress. Arch Intern Med. 2012; 172(21):1628-33. DOI: 10.1001/2013.jamainternmed.56. View

15.

McAlister F, van Diepen S, Padwal R, Johnson J, Majumdar S . How evidence-based are the recommendations in evidence-based guidelines?. PLoS Med. 2007; 4(8):e250. PMC: 1939859. DOI: 10.1371/journal.pmed.0040250. View

16.

Shea B, Hamel C, Wells G, Bouter L, Kristjansson E, Grimshaw J . AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009; 62(10):1013-20. DOI: 10.1016/j.jclinepi.2008.10.009. View

17.

Feuerstein J, Akbari M, Gifford A, Hurley C, Leffler D, Sheth S . Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 2014; 89(1):16-24. DOI: 10.1016/j.mayocp.2013.09.013. View

18.

Woodman J, Thomas J, Dickson K . How explicable are differences between reviews that appear to address a similar research question? A review of reviews of physical activity interventions. Syst Rev. 2012; 1:37. PMC: 3482390. DOI: 10.1186/2046-4053-1-37. View

19.

Dreier M, Borutta B, Stahmeyer J, Krauth C, Walter U . Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany. GMS Health Technol Assess. 2011; 6:Doc07. PMC: 3010881. DOI: 10.3205/hta000085. View

20.

Hussain T, Michel G, Shiffman R . The Yale Guideline Recommendation Corpus: a representative sample of the knowledge content of guidelines. Int J Med Inform. 2009; 78(5):354-63. DOI: 10.1016/j.ijmedinf.2008.11.001. View