A Manifesto for Reproducible Science

Overview

Journal Nat Hum Behav

Specialties Psychology
Social Sciences

Date 2021 May 6

PMID 33954258

Citations 706

Authors

Marcus R Munafo

Brian A Nosek

Dorothy V M Bishop

Katherine S Button

Christopher D Chambers

Nathalie Percie du Sert

Uri Simonsohn

Eric-Jan Wagenmakers

Jennifer J Ware

John P A Ioannidis

Affiliations

Soon will be listed here.

Abstract

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

Citing Articles

The role of art knowledge training on aesthetic judgements and executive functions.

Bara I, Cross E, Ramsey R R Soc Open Sci. 2025; 12(2):240175.

PMID: 40012756 PMC: 11858790. DOI: 10.1098/rsos.240175.

No evidence for decision fatigue using large-scale field data from healthcare.

Andersson D, Lindberg M, Tinghog G, Persson E Commun Psychol. 2025; 3(1):33.

PMID: 40011733 PMC: 11865449. DOI: 10.1038/s44271-025-00207-8.

Bias in data-driven replicability analysis of univariate brain-wide association studies.

Burns C, Fracasso A, Rousselet G Sci Rep. 2025; 15(1):6105.

PMID: 39972033 PMC: 11840108. DOI: 10.1038/s41598-025-89257-w.

Role, function, and expectations of research funding committees: Perspectives from committee members.

Blatch-Jones A, Boxall C, Meadmore K F1000Res. 2025; 13:1066.

PMID: 39959436 PMC: 11826072. DOI: 10.12688/f1000research.154665.2.

Dataset for a qualitative interview study exploring the barriers and facilitators to using and recommending aspirin for cancer prevention.

Lloyd K, Smith S Health Psychol Behav Med. 2025; 13(1):2463916.

PMID: 39959432 PMC: 11827028. DOI: 10.1080/21642850.2025.2463916.

References

Carp J . On the plurality of (methodological) worlds: estimating the analytic flexibility of FMRI experiments. Front Neurosci. 2012; 6:149. PMC: 3468892. DOI: 10.3389/fnins.2012.00149. View

Simonsohn U, Nelson L, Simmons J . P-curve: a key to the file-drawer. J Exp Psychol Gen. 2013; 143(2):534-47. DOI: 10.1037/a0033242. View

Sterne J, Davey Smith G . Sifting the evidence-what's wrong with significance tests?. BMJ. 2001; 322(7280):226-31. PMC: 1119478. DOI: 10.1136/bmj.322.7280.226. View

Franco A, Malhotra N, Simonovits G . Social science. Publication bias in the social sciences: unlocking the file drawer. Science. 2014; 345(6203):1502-5. DOI: 10.1126/science.1255484. View

Chambers C . Registered reports: a new publishing initiative at Cortex. Cortex. 2013; 49(3):609-10. DOI: 10.1016/j.cortex.2012.12.016. View

Walsh E, Rooney M, Appleby L, Wilkinson G . Open peer review: a randomised controlled trial. Br J Psychiatry. 2000; 176:47-51. DOI: 10.1192/bjp.176.1.47. View

Paneth N . Assessing the contributions of John Snow to epidemiology: 150 years after removal of the broad street pump handle. Epidemiology. 2004; 15(5):514-6. DOI: 10.1097/01.ede.0000135915.94799.00. View

Berg J, Bhalla N, Bourne P, Chalfie M, Drubin D, Fraser J . SCIENTIFIC COMMUNITY. Preprints for the life sciences. Science. 2016; 352(6288):899-901. DOI: 10.1126/science.aaf9133. View

Begley C, Ioannidis J . Reproducibility in science: improving the standard for basic and preclinical research. Circ Res. 2015; 116(1):116-26. DOI: 10.1161/CIRCRESAHA.114.303819. View

10.

Glasziou P, Altman D, Bossuyt P, Boutron I, Clarke M, Julious S . Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014; 383(9913):267-76. DOI: 10.1016/S0140-6736(13)62228-X. View

11.

Bath P, MacLeod M, Green A . Emulating multicentre clinical stroke trials: a new paradigm for studying novel interventions in experimental models of stroke. Int J Stroke. 2009; 4(6):471-9. DOI: 10.1111/j.1747-4949.2009.00386.x. View

12.

Button K, Ioannidis J, Mokrysz C, Nosek B, Flint J, Robinson E . Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013; 14(5):365-76. DOI: 10.1038/nrn3475. View

13.

Gotzsche P, Ioannidis J . Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses?. BMJ. 2012; 345:e7031. DOI: 10.1136/bmj.e7031. View

14.

Etter J, Burri M, Stapleton J . The impact of pharmaceutical company funding on results of randomized trials of nicotine replacement therapy for smoking cessation: a meta-analysis. Addiction. 2007; 102(5):815-22. DOI: 10.1111/j.1360-0443.2007.01822.x. View

15.

Smaldino P, McElreath R . The natural selection of bad science. R Soc Open Sci. 2016; 3(9):160384. PMC: 5043322. DOI: 10.1098/rsos.160384. View

16.

Liberati A, Altman D, Tetzlaff J, Mulrow C, Gotzsche P, Ioannidis J . The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009; 6(7):e1000100. PMC: 2707010. DOI: 10.1371/journal.pmed.1000100. View

17.

Ioannidis J . Why most published research findings are false. PLoS Med. 2005; 2(8):e124. PMC: 1182327. DOI: 10.1371/journal.pmed.0020124. View

18.

Wicherts J, Borsboom D, Kats J, Molenaar D . The poor availability of psychological research data for reanalysis. Am Psychol. 2006; 61(7):726-8. DOI: 10.1037/0003-066X.61.7.726. View

19.

Cohen J . The statistical power of abnormal-social psychological research: a review. J Abnorm Soc Psychol. 1962; 65:145-53. DOI: 10.1037/h0045186. View

20.

de Groot A . The meaning of "significance" for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der.... Acta Psychol (Amst). 2014; 148:188-94. DOI: 10.1016/j.actpsy.2014.02.001. View