Time-to-update of Systematic Reviews Relative to the Availability of New Evidence

Overview

Journal Syst Rev

Publisher Biomed Central

Date 2018 Nov 19

PMID 30447694

Citations 16

Authors

Rabia Bashir

Didi Surian

Adam G Dunn

Affiliations

Soon will be listed here.

Abstract

Background: A number of methods for deciding when a systematic review should be updated have been proposed, yet little is known about whether systematic reviews are updated more quickly when new evidence becomes available. Our aim was to examine the timing of systematic review updates relative to the availability of new evidence.

Methods: We performed a retrospective analysis of the update timing of systematic reviews published in the Cochrane Database of Systematic Reviews in 2010 relative to the availability of new trial evidence. We compared the update timing of systematic reviews with and without signals defined by the completion or publication of studies that were included in the updates.

Results: We found 43% (293/682) systematic reviews were updated before June 2017, of which 204 included an updated primary outcome meta-analysis (median update time 35.4 months; IQR 25.5-54.0), 38% (77/204) added new trials, and 4% (8/204) reported a change in conclusion. In the 171 systematic reviews with reconcilable trial reporting information, we did not find a clear difference in update timing (p = 0.05) between the 15 systematic reviews with a publication signal (median 25.3 months; IQR 15.3-43.5) and the 156 systematic reviews without a publication signal (median 34.4 months; IQR 25.1-52.2). In the 145 systematic reviews with reconcilable trial completion information, we did not find a difference in update timing (p = 0.33) between the 15 systematic reviews with a trial completion signal (median 26.0 months; IQR 19.3-49.5) and the 130 systematic reviews without a trial completion signal (median 32.4 months; IQR 24.1 to 46.0).

Conclusion: A minority of 2010 Cochrane reviews were updated before June 2017 to incorporate evidence from new primary studies, and very few updates led to a change in conclusion. We did not find clear evidence that updates were undertaken faster when new evidence was made available. New approaches for finding early signals that a systematic review conclusion is at risk of change may be useful in allocated resources to the updating of systematic reviews.

Citing Articles

How much can we save by applying artificial intelligence in evidence synthesis? Results from a pragmatic review to quantify workload efficiencies and cost savings.

Abogunrin S, Muir J, Zerbini C, Sarri G Front Pharmacol. 2025; 16:1454245.

PMID: 39959426 PMC: 11826052. DOI: 10.3389/fphar.2025.1454245.

Publicly available continuously updated topic specific databases of randomised clinical trials: A scoping review.

Boesen K, Hemkens L, Janiaud P, Hirt J medRxiv. 2024; .

PMID: 39606403 PMC: 11601726. DOI: 10.1101/2024.11.18.24317477.

Patient profiled data for treatment decision-making: valuable as an add-on to hepatitis C clinical guidelines?.

Brakenhoff S, Theijse T, van Wijngaarden P, Trautwein C, Brozat J, Tacke F BMC Med Inform Decis Mak. 2024; 24(1):227.

PMID: 39138441 PMC: 11321176. DOI: 10.1186/s12911-024-02608-x.

Measurement and outcomes of co-production in health and social care: a systematic review of empirical studies.

Nordin A, Kjellstrom S, Robert G, Masterson D, Areskoug Josefsson K BMJ Open. 2023; 13(9):e073808.

PMID: 37739472 PMC: 10533672. DOI: 10.1136/bmjopen-2023-073808.

The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews.

Hair K, Bahor Z, Macleod M, Liao J, Sena E BMC Biol. 2023; 21(1):189.

PMID: 37674179 PMC: 10483700. DOI: 10.1186/s12915-023-01686-z.

References

Jefferson T, Doshi P, Thompson M, Heneghan C . Ensuring safe and effective drugs: who can do what it takes?. BMJ. 2011; 342:c7258. DOI: 10.1136/bmj.c7258. View

Song S, Koo D, Jung S, Kang W, Kim E . The significance of the trial outcome was associated with publication rate and time to publication. J Clin Epidemiol. 2017; 84:78-84. DOI: 10.1016/j.jclinepi.2017.02.009. View

Shojania K, Sampson M, Ansari M, Ji J, Doucette S, Moher D . How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007; 147(4):224-33. DOI: 10.7326/0003-4819-147-4-200708210-00179. View

Atkins D, Fink K, Slutsky J . Better information for better health care: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality. Ann Intern Med. 2005; 142(12 Pt 2):1035-41. DOI: 10.7326/0003-4819-142-12_part_2-200506211-00002. View

Garner P, Hopewell S, Chandler J, MacLehose H, Akl E, Beyene J . When and how to update systematic reviews: consensus and checklist. BMJ. 2016; 354:i3507. PMC: 4955793. DOI: 10.1136/bmj.i3507. View

Pattanittum P, Laopaiboon M, Moher D, Lumbiganon P, Ngamjarus C . A comparison of statistical methods for identifying out-of-date systematic reviews. PLoS One. 2012; 7(11):e48894. PMC: 3502410. DOI: 10.1371/journal.pone.0048894. View

Jaidee W, Moher D, Laopaiboon M . Time to update and quantitative changes in the results of cochrane pregnancy and childbirth reviews. PLoS One. 2010; 5(7):e11553. PMC: 2903478. DOI: 10.1371/journal.pone.0011553. View

Garritty C, Tsertsvadze A, Tricco A, Sampson M, Moher D . Updating systematic reviews: an international survey. PLoS One. 2010; 5(4):e9914. PMC: 2848577. DOI: 10.1371/journal.pone.0009914. View

Dwan K, Gamble C, Williamson P, Kirkham J . Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013; 8(7):e66844. PMC: 3702538. DOI: 10.1371/journal.pone.0066844. View

10.

Moher D, Tsertsvadze A, Tricco A, Eccles M, Grimshaw J, Sampson M . When and how to update systematic reviews. Cochrane Database Syst Rev. 2008; (1):MR000023. PMC: 8941847. DOI: 10.1002/14651858.MR000023.pub3. View

11.

Shekelle P, Motala A, Johnsen B, Newberry S . Assessment of a method to detect signals for updating systematic reviews. Syst Rev. 2014; 3:13. PMC: 3937021. DOI: 10.1186/2046-4053-3-13. View

12.

Chung M, Newberry S, Ansari M, Yu W, Wu H, Lee J . Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol. 2012; 65(6):660-8. PMC: 4141462. DOI: 10.1016/j.jclinepi.2011.12.004. View

13.

French S, McDonald S, McKenzie J, Green S . Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?. BMC Med Res Methodol. 2005; 5:33. PMC: 1274326. DOI: 10.1186/1471-2288-5-33. View

14.

Page M, Shamseer L, Altman D, Tetzlaff J, Sampson M, Tricco A . Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016; 13(5):e1002028. PMC: 4878797. DOI: 10.1371/journal.pmed.1002028. View

15.

Peterson K, McDonagh M, Fu R . Decisions to update comparative drug effectiveness reviews vary based on type of new evidence. J Clin Epidemiol. 2011; 64(9):977-84. DOI: 10.1016/j.jclinepi.2010.11.019. View

16.

Sampson M, Shojania K, Garritty C, Horsley T, Ocampo M, Moher D . Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008; 61(6):531-6. DOI: 10.1016/j.jclinepi.2008.02.004. View

17.

Dechartres A, Ravaud P, Atal I, Riveros C, Boutron I . Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med. 2016; 14(1):100. PMC: 4932748. DOI: 10.1186/s12916-016-0639-x. View

18.

Gotzsche P . Why we need a broad perspective on meta-analysis. It may be crucially important for patients. BMJ. 2000; 321(7261):585-6. PMC: 1118486. DOI: 10.1136/bmj.321.7261.585. View

19.

Takwoingi Y, Hopewell S, Tovey D, Sutton A . A multicomponent decision tool for prioritising the updating of systematic reviews. BMJ. 2013; 347:f7191. DOI: 10.1136/bmj.f7191. View

20.

Bastian H, Glasziou P, Chalmers I . Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med. 2010; 7(9):e1000326. PMC: 2943439. DOI: 10.1371/journal.pmed.1000326. View