Development of AMSTAR: a Measurement Tool to Assess the Methodological Quality of Systematic Reviews

Overview

Journal BMC Med Res Methodol

Publisher Biomed Central

Specialties General Medicine
Health Services

Date 2007 Feb 17

PMID 17302989

Citations 1501

Authors

Beverley J Shea

Jeremy M Grimshaw

George A Wells

Maarten Boers

Neil Andersson

Candyce Hamel

Ashley C Porter

Peter Tugwell

David Moher

Lex M Bouter

Affiliations

Soon will be listed here.

Abstract

Background: Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.

Methods: A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.

Results: The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.

Conclusion: A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.

Citing Articles

Real-Time Navigation in Liver Surgery Through Indocyanine Green Fluorescence: An Updated Analysis of Worldwide Protocols and Applications.

Avella P, Spiezia S, Rotondo M, Cappuccio M, Scacchi A, Inglese G Cancers (Basel). 2025; 17(5).

PMID: 40075718 PMC: 11898688. DOI: 10.3390/cancers17050872.

Robotic-assisted versus laparoscopic adrenalectomy for large adrenal tumors: a systematic review and meta-analysis.

Zhang S, Chen C, Mo C, Pei Z, Dong Z, Ning Z Int Urol Nephrol. 2025; .

PMID: 40069536 DOI: 10.1007/s11255-025-04442-0.

Anatomic versus nonanatomic resection for intrahepatic cholangiocarcinoma: a systematic review and meta-analysis.

Jiang C, Hou G, Zhang Z, Qiang Z, Wang H, Zhou J Int J Surg. 2025; 111(1):1440-1453.

PMID: 40053809 PMC: 11745696. DOI: 10.1097/JS9.0000000000002134.

Radioactive stent versus normal stent insertion for inoperable malignant biliary obstruction: a systematic review and meta-analysis.

Chen X, Li J, Wan W, Zhang H, Xiong D Surg Endosc. 2025; .

PMID: 40047865 DOI: 10.1007/s00464-025-11571-1.

Quality indicators for the rehabilitation before and after total knee arthroplasty in Japan: a modified Delphi method and practice test.

Hiyama Y, Taniguchi M, Ohtera S, Wada O, Tanaka S, Kako M Arthroplasty. 2025; 7(1):11.

PMID: 40045402 PMC: 11884041. DOI: 10.1186/s42836-024-00297-5.

References

Jadad A, Moher M, Browman G, Booker L, Sigouin C, Fuentes M . Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ. 2000; 320(7234):537-40. PMC: 27295. DOI: 10.1136/bmj.320.7234.537. View

Moher D, Soeken K, Sampson M, Ben-Porat L, Berman B . Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr. 2002; 2:3. PMC: 99047. DOI: 10.1186/1471-2431-2-3. View

Downs S, Black N . The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998; 52(6):377-84. PMC: 1756728. DOI: 10.1136/jech.52.6.377. View

Sacks H, Berrier J, Reitman D, CHALMERS T . Meta-analyses of randomized controlled trials. N Engl J Med. 1987; 316(8):450-5. DOI: 10.1056/NEJM198702193160806. View

Verhagen A, de Vet H, de Bie R, Kessels A, Boers M, Bouter L . The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1999; 51(12):1235-41. DOI: 10.1016/s0895-4356(98)00131-0. View

Dickersin K . How important is publication bias? A synthesis of available data. AIDS Educ Prev. 1997; 9(1 Suppl):15-21. View

Moher D, Pham B, Klassen T, Schulz K, Berlin J, Jadad A . What contributions do languages other than English make on the results of meta-analyses?. J Clin Epidemiol. 2000; 53(9):964-72. DOI: 10.1016/s0895-4356(00)00188-8. View

Egger M, Schneider M, Junker C, Lengeler C, Antes G . Language bias in randomised controlled trials published in English and German. Lancet. 1997; 350(9074):326-9. DOI: 10.1016/S0140-6736(97)02419-7. View

Moher D, Cook D, Jadad A, Tugwell P, Moher M, Jones A . Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess. 1999; 3(12):i-iv, 1-98. View

10.

Stern J, Simes R . Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ. 1997; 315(7109):640-5. PMC: 2127436. DOI: 10.1136/bmj.315.7109.640. View

11.

Sterne J, Gavaghan D, Egger M . Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol. 2000; 53(11):1119-29. DOI: 10.1016/s0895-4356(00)00242-0. View

12.

Oxman A . Checklists for review articles. BMJ. 1994; 309(6955):648-51. PMC: 2541456. DOI: 10.1136/bmj.309.6955.648. View

13.

Davidoff F, Haynes B, Sackett D, Smith R . Evidence based medicine. BMJ. 1995; 310(6987):1085-6. PMC: 2549494. DOI: 10.1136/bmj.310.6987.1085. View

14.

McDowell I, Jenkinson C . Development standards for health measures. J Health Serv Res Policy. 1996; 1(4):238-46. DOI: 10.1177/135581969600100410. View

15.

Lau J, Ioannidis J, Schmid C . Summing up evidence: one answer is not always enough. Lancet. 1998; 351(9096):123-7. DOI: 10.1016/S0140-6736(97)08468-7. View

16.

Hopewell S, McDonald S, Clarke M, Egger M . Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007; (2):MR000010. PMC: 8973936. DOI: 10.1002/14651858.MR000010.pub3. View

17.

Sutton A, Duval S, Tweedie R, Abrams K, Jones D . Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000; 320(7249):1574-7. PMC: 27401. DOI: 10.1136/bmj.320.7249.1574. View

18.

Jadad A, Moore R, Carroll D, Jenkinson C, Reynolds D, Gavaghan D . Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996; 17(1):1-12. DOI: 10.1016/0197-2456(95)00134-4. View

19.

Lau J, Ioannidis J, Terrin N, Schmid C, Olkin I . The case of the misleading funnel plot. BMJ. 2006; 333(7568):597-600. PMC: 1570006. DOI: 10.1136/bmj.333.7568.597. View

20.

Dickersin K . The existence of publication bias and risk factors for its occurrence. JAMA. 1990; 263(10):1385-9. View