From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics

Overview

Journal PLoS One

Specialties General Medicine
Science

Date 2015 Jul 9

PMID 26154165

Citations 12

Authors

Alejandra Gonzalez-Beltran

Peter Li

Jun Zhao

Maria Susana Avila-Garcia

Marco Roos

Mark Thompson

Eelke van der Horst

Rajaram Kaliyaperumal

Ruibang Luo

Tin-Lap Lee

Tak-Wah Lam

Scott C Edmunds

Susanna-Assunta Sansone

Philippe Rocca-Serra

Affiliations

Soon will be listed here.

Abstract

Motivation: Reproducing the results from a scientific paper can be challenging due to the absence of data and the computational tools required for their analysis. In addition, details relating to the procedures used to obtain the published results can be difficult to discern due to the use of natural language when reporting how experiments have been performed. The Investigation/Study/Assay (ISA), Nanopublications (NP), and Research Objects (RO) models are conceptual data modelling frameworks that can structure such information from scientific papers. Computational workflow platforms can also be used to reproduce analyses of data in a principled manner. We assessed the extent by which ISA, NP, and RO models, together with the Galaxy workflow system, can capture the experimental processes and reproduce the findings of a previously published paper reporting on the development of SOAPdenovo2, a de novo genome assembler.

Results: Executable workflows were developed using Galaxy, which reproduced results that were consistent with the published findings. A structured representation of the information in the SOAPdenovo2 paper was produced by combining the use of ISA, NP, and RO models. By structuring the information in the published paper using these data and scientific workflow modelling frameworks, it was possible to explicitly declare elements of experimental design, variables, and findings. The models served as guides in the curation of scientific information and this led to the identification of inconsistencies in the original published paper, thereby allowing its authors to publish corrections in the form of an errata.

Availability: SOAPdenovo2 scripts, data, and results are available through the GigaScience Database: http://dx.doi.org/10.5524/100044; the workflows are available from GigaGalaxy: http://galaxy.cbiit.cuhk.edu.hk; and the representations using the ISA, NP, and RO models are available through the SOAPdenovo2 case study website http://isa-tools.github.io/soapdenovo2/.

Contact: philippe.rocca-serra@oerc.ox.ac.uk and susanna-assunta.sansone@oerc.ox.ac.uk.

Citing Articles

Assessing computational reproducibility in Behavior Research Methods.

Ellis D, Towse J, Brown O, Cork A, Davidson B, Devereux S Behav Res Methods. 2024; 56(8):8745-8760.

PMID: 39322919 PMC: 11525395. DOI: 10.3758/s13428-024-02501-5.

Scientific modelling can be accessible, interoperable and user friendly: A case study for pasture and livestock modelling in Spain.

Marquez Torres A, Balbi S, Villa F PLoS One. 2023; 18(2):e0281348.

PMID: 36827966 PMC: 9957615. DOI: 10.1371/journal.pone.0281348.

Recommendations for Reporting Machine Learning Analyses in Clinical Research.

Stevens L, Mortazavi B, Deo R, Curtis L, Kao D Circ Cardiovasc Qual Outcomes. 2020; 13(10):e006556.

PMID: 33079589 PMC: 8320533. DOI: 10.1161/CIRCOUTCOMES.120.006556.

AVOCADO: Visualization of Workflow-Derived Data Provenance for Reproducible Biomedical Research.

Stitz H, Luger S, Streit M, Gehlenborg N Comput Graph Forum. 2018; 35(3):481-490.

PMID: 29973745 PMC: 6027754. DOI: 10.1111/cgf.12924.

Looking back: forward looking.

Edmunds S, Nogoy N, Zauner H, Li P, Hunter C, Zhe X Gigascience. 2017; 6(9):1-3.

PMID: 28938718 PMC: 5603759. DOI: 10.1093/gigascience/gix064.

References

Sansone S, Rocca-Serra P, Field D, Maguire E, Taylor C, Hofmann O . Toward interoperable bioscience data. Nat Genet. 2012; 44(2):121-6. PMC: 3428019. DOI: 10.1038/ng.1054. View

Begley C, Ellis L . Drug development: Raise standards for preclinical cancer research. Nature. 2012; 483(7391):531-3. DOI: 10.1038/483531a. View

Kohonen P, Benfenati E, Bower D, Ceder R, Crump M, Cross K . The ToxBank Data Warehouse: Supporting the Replacement of In Vivo Repeated Dose Systemic Toxicity Testing. Mol Inform. 2016; 32(1):47-63. DOI: 10.1002/minf.201200114. View

Oinn T, Addis M, Ferris J, Marvin D, Senger M, Greenwood M . Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics. 2004; 20(17):3045-54. DOI: 10.1093/bioinformatics/bth361. View

Mons B, van Haagen H, Chichester C, Hoen P, den Dunnen J, van Ommen G . The value of data. Nat Genet. 2011; 43(4):281-3. DOI: 10.1038/ng0411-281. View

Wang J, Wang W, Li R, Li Y, Tian G, Goodman L . The diploid genome sequence of an Asian individual. Nature. 2008; 456(7218):60-5. PMC: 2716080. DOI: 10.1038/nature07484. View

Ince D, Hatton L, Graham-Cumming J . The case for open computer programs. Nature. 2012; 482(7386):485-8. DOI: 10.1038/nature10836. View

Dumontier M, Baker C, Baran J, Callahan A, Chepelev L, Cruz-Toledo J . The Semanticscience Integrated Ontology (SIO) for biomedical research and knowledge discovery. J Biomed Semantics. 2014; 5(1):14. PMC: 4015691. DOI: 10.1186/2041-1480-5-14. View

. Enhancing reproducibility. Nat Methods. 2013; 10(5):367. DOI: 10.1038/nmeth.2471. View

10.

Ioannidis J, Allison D, Ball C, Coulibaly I, Cui X, Culhane A . Repeatability of published microarray gene expression analyses. Nat Genet. 2009; 41(2):149-55. DOI: 10.1038/ng.295. View

11.

Juty N, Le Novere N, Laibe C . Identifiers.org and MIRIAM Registry: community resources to provide persistent identification. Nucleic Acids Res. 2011; 40(Database issue):D580-6. PMC: 3245029. DOI: 10.1093/nar/gkr1097. View

12.

Kettner C, Field D, Sansone S, Taylor C, Aerts J, Binns N . Meeting Report from the Second "Minimum Information for Biological and Biomedical Investigations" (MIBBI) workshop. Stand Genomic Sci. 2011; 3(3):259-66. PMC: 3035314. DOI: 10.4056/sigs.147362. View

13.

Luo R, Liu B, Xie Y, Li Z, Huang W, Yuan J . SOAPdenovo2: an empirically improved memory-efficient short-read de novo assembler. Gigascience. 2013; 1(1):18. PMC: 3626529. DOI: 10.1186/2047-217X-1-18. View

14.

Cokol M, Ozbay F, Rodriguez-Esteban R . Retraction rates are on the rise. EMBO Rep. 2008; 9(1):2. PMC: 2246630. DOI: 10.1038/sj.embor.7401143. View

15.

Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W . The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007; 25(11):1251-5. PMC: 2814061. DOI: 10.1038/nbt1346. View

16.

Ribeiro F, Przybylski D, Yin S, Sharpe T, Gnerre S, Abouelleil A . Finished bacterial genomes from shotgun sequence data. Genome Res. 2012; 22(11):2270-7. PMC: 3483556. DOI: 10.1101/gr.141515.112. View

17.

Garijo D, Kinnings S, Xie L, Xie L, Zhang Y, Bourne P . Quantifying reproducibility in computational biology: the case of the tuberculosis drugome. PLoS One. 2013; 8(11):e80278. PMC: 3842296. DOI: 10.1371/journal.pone.0080278. View

18.

Vasilevsky N, Brush M, Paddock H, Ponting L, Tripathy S, LaRocca G . On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ. 2013; 1:e148. PMC: 3771067. DOI: 10.7717/peerj.148. View

19.

Smith R . Classical peer review: an empty gun. Breast Cancer Res. 2010; 12 Suppl 4:S13. PMC: 3005733. DOI: 10.1186/bcr2742. View

20.

Gonzalez-Beltran A, Neumann S, Maguire E, Sansone S, Rocca-Serra P . The Risa R/Bioconductor package: integrative data analysis from experimental metadata and back again. BMC Bioinformatics. 2014; 15 Suppl 1:S11. PMC: 4015122. DOI: 10.1186/1471-2105-15-S1-S11. View