Overinterpretation and Misreporting of Prognostic Factor Studies in Oncology: a Systematic Review

Overview

Journal Br J Cancer

Specialty Oncology

Date 2018 Oct 25

PMID 30353050

Citations 12

Authors

Emmanuelle Kempf

Jennifer A de Beyer

Jonathan Cook

Jane Holmes

Seid Mohammed

Tri-Long Nguyen

Iveta Simera

Marialena Trivella

Douglas G Altman

Sally Hopewell

Karel G M Moons

Raphael Porcher

Johannes B Reitsma

Willi Sauerbrei

Gary S Collins

Affiliations

Soon will be listed here.

Abstract

Background: Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology.

Methods: This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty).

Results: Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1-Q3, 1-7). The prognostic factors' effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions.

Conclusions: Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.

Citing Articles

An overview of statistical methods for biomarkers relevant to early clinical development of cancer immunotherapies.

Dejardin D, Kraxner A, Schindler E, Stadler N, Wolbers M Front Immunol. 2024; 15:1351584.

PMID: 39234243 PMC: 11371698. DOI: 10.3389/fimmu.2024.1351584.

Linguistic spin in randomized controlled trials about age-related macular degeneration.

Veldhuis N, Nuijts M, Isphording L, Lee-Kong F, Imhof S, Stegeman I Front Epidemiol. 2024; 2:961996.

PMID: 38455287 PMC: 10910936. DOI: 10.3389/fepid.2022.961996.

Verification of nucleotide sequence reagent identities in original publications in high impact factor cancer research journals.

Pathmendra P, Park Y, Enguita F, Byrne J Naunyn Schmiedebergs Arch Pharmacol. 2024; 397(7):5049-5066.

PMID: 38194106 PMC: 11166861. DOI: 10.1007/s00210-023-02846-2.

Could palliative sedation be seen as unnamed euthanasia?: a survey among healthcare professionals in oncology.

Lucchi E, Milder M, Dardenne A, Bouleuc C BMC Palliat Care. 2023; 22(1):97.

PMID: 37468913 PMC: 10354970. DOI: 10.1186/s12904-023-01219-z.

REMARK guidelines for tumour biomarker study reporting: a remarkable history.

Hayes D, Sauerbrei W, McShane L Br J Cancer. 2022; 128(3):443-445.

PMID: 36476656 PMC: 9938190. DOI: 10.1038/s41416-022-02046-4.

References

Simon R, Altman D . Statistical aspects of prognostic factor studies in oncology. Br J Cancer. 1994; 69(6):979-85. PMC: 1969431. DOI: 10.1038/bjc.1994.192. View

Mack J, Weeks J, Wright A, Block S, Prigerson H . End-of-life discussions, goal attainment, and distress at the end of life: predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010; 28(7):1203-8. PMC: 2834470. DOI: 10.1200/JCO.2009.25.4672. View

Fletcher R, Black B . "Spin" in scientific writing: scientific mischief and legal jeopardy. Med Law. 2007; 26(3):511-25. View

Rennie D . Trial registration: a great idea switches from ignored to irresistible. JAMA. 2004; 292(11):1359-62. DOI: 10.1001/jama.292.11.1359. View

Freidlin B, McShane L, Polley M, Korn E . Randomized phase II trial designs with biomarkers. J Clin Oncol. 2012; 30(26):3304-9. PMC: 3434989. DOI: 10.1200/JCO.2012.43.3946. View

Hayden J, van der Windt D, Cartwright J, Cote P, Bombardier C . Assessing bias in studies of prognostic factors. Ann Intern Med. 2013; 158(4):280-6. DOI: 10.7326/0003-4819-158-4-201302190-00009. View

Prasad V, Jorgenson J, Ioannidis J, Cifu A . Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes. J Clin Epidemiol. 2013; 66(4):361-366.e4. DOI: 10.1016/j.jclinepi.2012.11.005. View

Vedula S, Goldman P, Rona I, Greene T, Dickersin K . Implementation of a publication strategy in the context of reporting biases. A case study based on new documents from Neurontin litigation. Trials. 2012; 13:136. PMC: 3439687. DOI: 10.1186/1745-6215-13-136. View

Mallett S, Timmer A, Sauerbrei W, Altman D . Reporting of prognostic studies of tumour markers: a review of published articles in relation to REMARK guidelines. Br J Cancer. 2009; 102(1):173-80. PMC: 2795163. DOI: 10.1038/sj.bjc.6605462. View

10.

Unger J, Barlow W, Ramsey S, LeBlanc M, Blanke C, Hershman D . The Scientific Impact of Positive and Negative Phase 3 Cancer Clinical Trials. JAMA Oncol. 2016; 2(7):875-81. PMC: 4945370. DOI: 10.1001/jamaoncol.2015.6487. View

11.

Kyzas P, Denaxa-Kyza D, Ioannidis J . Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer. 2007; 43(17):2559-79. DOI: 10.1016/j.ejca.2007.08.030. View

12.

Ioannidis J, Panagiotou O . Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. JAMA. 2011; 305(21):2200-10. DOI: 10.1001/jama.2011.713. View

13.

Chan A . Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med. 2008; 5(11):e230. PMC: 2586359. DOI: 10.1371/journal.pmed.0050230. View

14.

Boutron I, Dutton S, Ravaud P, Altman D . Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010; 303(20):2058-64. DOI: 10.1001/jama.2010.651. View

15.

Kyzas P, Loizou K, Ioannidis J . Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005; 97(14):1043-55. DOI: 10.1093/jnci/dji184. View

16.

Riley R, Hayden J, Steyerberg E, Moons K, Abrams K, Kyzas P . Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013; 10(2):e1001380. PMC: 3564757. DOI: 10.1371/journal.pmed.1001380. View

17.

Berger V . Spin and bias: the tip of the iceberg. Ann Oncol. 2013; 24(7):1949. PMC: 3697056. DOI: 10.1093/annonc/mdt189. View

18.

Hagerty R, Butow P, Ellis P, Lobb E, Pendlebury S, Leighl N . Communicating with realism and hope: incurable cancer patients' views on the disclosure of prognosis. J Clin Oncol. 2005; 23(6):1278-88. DOI: 10.1200/JCO.2005.11.138. View

19.

Greenhalgh T . Commentary: scientific heads are not turned by rhetoric. BMJ. 1995; 310(6985):987-8. PMC: 2549364. DOI: 10.1136/bmj.310.6985.987. View

20.

Gewandter J, McKeown A, McDermott M, Dworkin J, Smith S, Gross R . Data interpretation in analgesic clinical trials with statistically nonsignificant primary analyses: an ACTTION systematic review. J Pain. 2014; 16(1):3-10. DOI: 10.1016/j.jpain.2014.10.003. View