The Systematic Review Toolbox: Keeping Up to Date with Tools to Support Evidence Synthesis

Overview

Journal Syst Rev

Publisher Biomed Central

Date 2022 Dec 2

PMID 36457048

Authors

Eugenie Evelynne Johnson

Hannah OKeefe

Anthea Sutton

Christopher Marshall

Affiliations

Soon will be listed here.

Abstract

Background: The Systematic Review (SR) Toolbox was developed in 2014 to collate tools that can be used to support the systematic review process. Since its inception, the breadth of evidence synthesis methodologies has expanded greatly. This work describes the process of updating the SR Toolbox in 2022 to reflect these changes in evidence synthesis methodology. We also briefly analysed included tools and guidance to identify any potential gaps in what is currently available to researchers.

Methods: We manually extracted all guidance and software tools contained within the SR Toolbox in February 2022. A single reviewer, with a second checking a proportion, extracted and analysed information from records contained within the SR Toolbox using Microsoft Excel. Using this spreadsheet and Microsoft Access, the SR Toolbox was updated to reflect expansion of evidence synthesis methodologies and brief analysis conducted.

Results: The updated version of the SR Toolbox was launched on 13 May 2022, with 235 software tools and 112 guidance documents included. Regarding review families, most software tools (N = 223) and guidance documents (N = 78) were applicable to systematic reviews. However, there were fewer tools and guidance documents applicable to reviews of reviews (N = 66 and N = 22, respectively), while qualitative reviews were less served by guidance documents (N = 19). In terms of review production stages, most guidance documents surrounded quality assessment (N = 70), while software tools related to searching and synthesis (N = 84 and N = 82, respectively). There appears to be a paucity of tools and guidance relating to stakeholder engagement (N = 2 and N = 3, respectively).

Conclusions: The SR Toolbox provides a platform for those undertaking evidence syntheses to locate guidance and software tools to support different aspects of the review process across multiple review types. However, this work has also identified potential gaps in guidance and software that could inform future research.

Citing Articles

Publicly available continuously updated topic specific databases of randomised clinical trials: A scoping review.

Boesen K, Hemkens L, Janiaud P, Hirt J medRxiv. 2024; .

PMID: 39606403 PMC: 11601726. DOI: 10.1101/2024.11.18.24317477.

Usability of the "Systematic Review Support" computer system: a methodological study.

Escaldelai F, Escaldelai L, Bergamaschi D Rev Lat Am Enfermagem. 2024; 32:e4374.

PMID: 39476140 PMC: 11526212. DOI: 10.1590/1518-8345.7081.4374.

: a large language model that generates search queries for systematic reviews.

Adam G, DeYoung J, Paul A, Saldanha I, Balk E, Trikalinos T JAMIA Open. 2024; 7(3):ooae098.

PMID: 39323560 PMC: 11424077. DOI: 10.1093/jamiaopen/ooae098.

(Semi)automated approaches to data extraction for systematic reviews and meta-analyses in social sciences: A living review.

Legate A, Nimon K, Noblin A F1000Res. 2024; 13:664.

PMID: 39220382 PMC: 11364972. DOI: 10.12688/f1000research.151493.1.

An Automated Literature Review Tool (LiteRev) for Streamlining and Accelerating Research Using Natural Language Processing and Machine Learning: Descriptive Performance Evaluation Study.

Orel E, Ciglenecki I, Thiabaud A, Temerev A, Calmy A, Keiser O J Med Internet Res. 2023; 25:e39736.

PMID: 37713261 PMC: 10541641. DOI: 10.2196/39736.

References

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T . Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019; 8(1):325. PMC: 6911272. DOI: 10.1186/s13643-019-1248-5. View

Johnson E, OKeefe H, Sutton A, Marshall C . The Systematic Review Toolbox: keeping up to date with tools to support evidence synthesis. Syst Rev. 2022; 11(1):258. PMC: 9713957. DOI: 10.1186/s13643-022-02122-z. View

Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z . What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018; 18(1):5. PMC: 5761190. DOI: 10.1186/s12874-017-0468-4. View

Damen J, Hooft L . The increasing need for systematic reviews of prognosis studies: strategies to facilitate review production and improve quality of primary research. Diagn Progn Res. 2019; 3:2. PMC: 6460843. DOI: 10.1186/s41512-019-0049-6. View

Miake-Lye I, Hempel S, Shanman R, Shekelle P . What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016; 5:28. PMC: 4750281. DOI: 10.1186/s13643-016-0204-x. View

Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S . Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews. J Health Serv Res Policy. 2019; 24(4):245-255. DOI: 10.1177/1355819619841647. View

Richter B, Hemmingsen B, Metzendorf M, Takwoingi Y . Development of type 2 diabetes mellitus in people with intermediate hyperglycaemia. Cochrane Database Syst Rev. 2018; 10:CD012661. PMC: 6516891. DOI: 10.1002/14651858.CD012661.pub2. View

Elliott J, Turner T, Clavisi O, Thomas J, Higgins J, Mavergames C . Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 2014; 11(2):e1001603. PMC: 3928029. DOI: 10.1371/journal.pmed.1001603. View

Petropoulou M, Nikolakopoulou A, Veroniki A, Rios P, Vafaei A, Zarin W . Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2016; 82:20-28. DOI: 10.1016/j.jclinepi.2016.11.002. View

10.

Arno A, Elliott J, Wallace B, Turner T, Thomas J . The views of health guideline developers on the use of automation in health evidence synthesis. Syst Rev. 2021; 10(1):16. PMC: 7796617. DOI: 10.1186/s13643-020-01569-2. View

11.

Sutton A, Clowes M, Preston L, Booth A . Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019; 36(3):202-222. DOI: 10.1111/hir.12276. View

12.

Westby M, Dumville J, Stubbs N, Norman G, Wong J, Cullum N . Protease activity as a prognostic factor for wound healing in venous leg ulcers. Cochrane Database Syst Rev. 2018; 9:CD012841. PMC: 6513613. DOI: 10.1002/14651858.CD012841.pub2. View

13.

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S . GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017; 358:j3453. PMC: 5539518. DOI: 10.1136/bmj.j3453. View

14.

Snilstveit B, Vojtkova M, Bhavsar A, Stevenson J, Gaarder M . Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas. J Clin Epidemiol. 2016; 79:120-129. DOI: 10.1016/j.jclinepi.2016.05.015. View

15.

Tricco A, Lillie E, Zarin W, OBrien K, Colquhoun H, Kastner M . A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016; 16:15. PMC: 4746911. DOI: 10.1186/s12874-016-0116-4. View