The Reproducibility Crisis in the Age of Digital Medicine

Overview

Journal NPJ Digit Med

Specialty Medical Informatics

Date 2019 Jul 16

PMID 31304352

Citations 70

Authors

Aaron Stupple

David Singerman

Leo Anthony Celi

Affiliations

Soon will be listed here.

Citing Articles

Data-Driven Early Prediction of Cerebral Palsy Using AutoML and interpretable kinematic features.

Segado M, Prosser L, Duncan A, Johnson M, Kording K medRxiv. 2025; .

PMID: 39990562 PMC: 11844582. DOI: 10.1101/2025.02.10.25322007.

Glucagon-Like Peptide 1 Receptor Agonists and Chronic Lower Respiratory Disease Among Type 2 Diabetes Patients: Replication and Reliability Assessment Across a Research Network.

Conover M, Albogami Y, Hardin J, Reich C, Ostropolets A, Ryan P Pharmacoepidemiol Drug Saf. 2025; 34(1):e70087.

PMID: 39805811 PMC: 11730806. DOI: 10.1002/pds.70087.

Strategies for creation of data reserve and stress testing of medical AI products.

Chen H, Tie Y, Cao X, Young G, Xu X BMJ Health Care Inform. 2024; 31(1).

PMID: 39632098 PMC: 11624794. DOI: 10.1136/bmjhci-2024-101184.

Deep learning-based Alzheimer's disease detection: reproducibility and the effect of modeling choices.

Turrisi R, Verri A, Barla A Front Comput Neurosci. 2024; 18:1360095.

PMID: 39371524 PMC: 11451303. DOI: 10.3389/fncom.2024.1360095.

ENCORE: a practical implementation to improve reproducibility and transparency of computational research.

van Kampen A, Mahamune U, Jongejan A, van Schaik B, Balashova D, Lashgari D Nat Commun. 2024; 15(1):8117.

PMID: 39284801 PMC: 11405857. DOI: 10.1038/s41467-024-52446-8.

References

Ferreira C, Bastille-Rousseau G, Bennett A, Ellington E, Terwissen C, Austin C . The evolution of peer review as a basis for scientific publication: directional selection towards a robust discipline?. Biol Rev Camb Philos Soc. 2015; 91(3):597-610. DOI: 10.1111/brv.12185. View

Gottesman O, Johansson F, Komorowski M, Faisal A, Sontag D, Doshi-Velez F . Guidelines for reinforcement learning in healthcare. Nat Med. 2019; 25(1):16-18. DOI: 10.1038/s41591-018-0310-5. View

Goodman S, Fanelli D, Ioannidis J . What does research reproducibility mean?. Sci Transl Med. 2016; 8(341):341ps12. DOI: 10.1126/scitranslmed.aaf5027. View

Baker M . 1,500 scientists lift the lid on reproducibility. Nature. 2016; 533(7604):452-4. DOI: 10.1038/533452a. View

Celi L, Citi L, Ghassemi M, Pollard T . The PLOS ONE collection on machine learning in health and biomedicine: Towards open code and open data. PLoS One. 2019; 14(1):e0210232. PMC: 6333339. DOI: 10.1371/journal.pone.0210232. View

Simmons J, Nelson L, Simonsohn U . False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011; 22(11):1359-66. DOI: 10.1177/0956797611417632. View

Lederer D, Bell S, Branson R, Chalmers J, Marshall R, Maslove D . Control of Confounding and Reporting of Results in Causal Inference Studies. Guidance for Authors from Editors of Respiratory, Sleep, and Critical Care Journals. Ann Am Thorac Soc. 2018; 16(1):22-28. DOI: 10.1513/AnnalsATS.201808-564PS. View

Csiszar A . Peer review: Troubled from the start. Nature. 2016; 532(7599):306-8. DOI: 10.1038/532306a. View

Abola M, Prasad V . The Use of Superlatives in Cancer Research. JAMA Oncol. 2015; 2(1):139-41. DOI: 10.1001/jamaoncol.2015.3931. View

10.

Allison D, Brown A, George B, Kaiser K . Reproducibility: A tragedy of errors. Nature. 2016; 530(7588):27-9. PMC: 4831566. DOI: 10.1038/530027a. View

11.

Vul E, Harris C, Winkielman P, Pashler H . Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition. Perspect Psychol Sci. 2015; 4(3):274-90. DOI: 10.1111/j.1745-6924.2009.01125.x. View

12.

Suls J, Martin R . The Air We Breathe: A Critical Look at Practices and Alternatives in the Peer-Review Process. Perspect Psychol Sci. 2015; 4(1):40-50. DOI: 10.1111/j.1745-6924.2009.01105.x. View

13.

. PSYCHOLOGY. Estimating the reproducibility of psychological science. Science. 2015; 349(6251):aac4716. DOI: 10.1126/science.aac4716. View