Publication and Other Reporting Biases in Cognitive Sciences: Detection, Prevalence, and Prevention

Overview

Journal Trends Cogn Sci

Date 2014 Mar 25

PMID 24656991

Citations 140

Authors

John P A Ioannidis

Marcus R Munafo

Paolo Fusar-Poli

Brian A Nosek

Sean P David

Affiliations

Soon will be listed here.

Abstract

Recent systematic reviews and empirical evaluations of the cognitive sciences literature suggest that publication and other reporting biases are prevalent across diverse domains of cognitive science. In this review, we summarize the various forms of publication and reporting biases and other questionable research practices, and overview the available methods for probing into their existence. We discuss the available empirical evidence for the presence of such biases across the neuroimaging, animal, other preclinical, psychological, clinical trials, and genetics literature in the cognitive sciences. We also highlight emerging solutions (from study design to data analyses and reporting) to prevent bias and improve the fidelity in the field of cognitive science research.

Citing Articles

Meta-analytic prospective associations between self-esteem and eating disorders appear to be spurious: a reanalysis and comment on Krauss et al. (2023).

Sorjonen K, Ghaderi A, Melin B Front Psychol. 2025; 16:1463701.

PMID: 39989636 PMC: 11842458. DOI: 10.3389/fpsyg.2025.1463701.

Improving research transparency with individualized report cards: A feasibility study in clinical trials at a large university medical center.

Franzen D, Salholz-Hillel M, Muller-Ohlraun S, Strech D BMC Med Res Methodol. 2025; 25(1):37.

PMID: 39948475 PMC: 11823227. DOI: 10.1186/s12874-025-02482-9.

Context familiarity is a third kind of episodic memory distinct from item familiarity and recollection.

Addante R, Clise E, Waechter R, Bengson J, Drane D, Perez-Caban J iScience. 2025; 27(12):111439.

PMID: 39758982 PMC: 11699256. DOI: 10.1016/j.isci.2024.111439.

The point of no return in the Emotional Stop-Signal Task: A matter of affect or method?.

Coccaro A, Maffei A, Kleffner K, Carolan P, Vallesi A, DAdamo G PLoS One. 2024; 19(12):e0315082.

PMID: 39637207 PMC: 11620632. DOI: 10.1371/journal.pone.0315082.

Neuroimaging-based variability in subtyping biomarkers for psychiatric heterogeneity.

Wen Z, Hammoud M, Siegel C, Laska E, Abu-Amara D, Etkin A Mol Psychiatry. 2024; .

PMID: 39511450 DOI: 10.1038/s41380-024-02807-y.

References

Cuijpers P, Turner E, Mohr D, Hofmann S, Andersson G, Berking M . Comparison of psychotherapies for adult depression to pill placebo control groups: a meta-analysis. Psychol Med. 2013; 44(4):685-95. DOI: 10.1017/S0033291713000457. View

Koole S, Lakens D . Rewarding Replications: A Sure and Simple Way to Improve Psychological Science. Perspect Psychol Sci. 2015; 7(6):608-14. DOI: 10.1177/1745691612462586. View

Lau J, Ioannidis J, Terrin N, Schmid C, Olkin I . The case of the misleading funnel plot. BMJ. 2006; 333(7568):597-600. PMC: 1570006. DOI: 10.1136/bmj.333.7568.597. View

Peters J, Sutton A, Jones D, Abrams K, Rushton L . Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006; 295(6):676-80. DOI: 10.1001/jama.295.6.676. View

Nosek B, Spies J, Motyl M . Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspect Psychol Sci. 2015; 7(6):615-31. PMC: 10540222. DOI: 10.1177/1745691612459058. View

Thompson S, Ekelund U, Jebb S, Lindroos A, Mander A, Sharp S . A proposed method of bias adjustment for meta-analyses of published observational studies. Int J Epidemiol. 2010; 40(3):765-77. PMC: 3147067. DOI: 10.1093/ije/dyq248. View

Turner E, Matthews A, Linardatos E, Tell R, Rosenthal R . Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008; 358(3):252-60. DOI: 10.1056/NEJMsa065779. View

Jennings R, Van Horn J . Publication bias in neuroimaging research: implications for meta-analyses. Neuroinformatics. 2011; 10(1):67-80. PMC: 4368431. DOI: 10.1007/s12021-011-9125-y. View

Begley C, Ellis L . Drug development: Raise standards for preclinical cancer research. Nature. 2012; 483(7391):531-3. DOI: 10.1038/483531a. View

10.

Giner-Sorolla R . Science or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science. Perspect Psychol Sci. 2015; 7(6):562-71. DOI: 10.1177/1745691612457576. View

11.

Evangelou E, Ioannidis J . Meta-analysis methods for genome-wide association studies and beyond. Nat Rev Genet. 2013; 14(6):379-89. DOI: 10.1038/nrg3472. View

12.

Poldrack R, Fletcher P, Henson R, Worsley K, Brett M, Nichols T . Guidelines for reporting an fMRI study. Neuroimage. 2008; 40(2):409-414. PMC: 2287206. DOI: 10.1016/j.neuroimage.2007.11.048. View

13.

Macleod M, van der Worp H, Sena E, Howells D, Dirnagl U, Donnan G . Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. Stroke. 2008; 39(10):2824-9. DOI: 10.1161/STROKEAHA.108.515957. View

14.

Button K, Ioannidis J, Mokrysz C, Nosek B, Flint J, Robinson E . Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013; 14(5):365-76. DOI: 10.1038/nrn3475. View

15.

Panagiotou O, Willer C, Hirschhorn J, Ioannidis J . The power of meta-analysis in genome-wide association studies. Annu Rev Genomics Hum Genet. 2013; 14:441-65. PMC: 4040957. DOI: 10.1146/annurev-genom-091212-153520. View

16.

Hirschhorn J . Genomewide association studies--illuminating biologic pathways. N Engl J Med. 2009; 360(17):1699-701. DOI: 10.1056/NEJMp0808934. View

17.

Etter J, Burri M, Stapleton J . The impact of pharmaceutical company funding on results of randomized trials of nicotine replacement therapy for smoking cessation: a meta-analysis. Addiction. 2007; 102(5):815-22. DOI: 10.1111/j.1360-0443.2007.01822.x. View

18.

Anderson M, Martinson B, De Vries R . Normative dissonance in science: results from a national survey of u.s. Scientists. J Empir Res Hum Res Ethics. 2009; 2(4):3-14. DOI: 10.1525/jer.2007.2.4.3. View

19.

Copas J, Jackson D . A bound for publication bias based on the fraction of unpublished studies. Biometrics. 2004; 60(1):146-53. DOI: 10.1111/j.0006-341X.2004.00161.x. View

20.

Ioannidis J, Trikalinos T . Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol. 2005; 58(6):543-9. DOI: 10.1016/j.jclinepi.2004.10.019. View