Review of Pragmatic Trials Found That Multiple Primary Outcomes Are Common but So Too Are Discrepancies Between Protocols and Final Reports

Overview

Journal J Clin Epidemiol

Publisher Elsevier

Specialty Public Health

Date 2021 Dec 13

PMID 34896234

Citations 3

Authors

Pascale Nevins

Shelley Vanderhout

Kelly Carroll

Stuart G Nicholls

Seana N Semchishen

Jamie C Brehaut

Dean A Fergusson

Bruno Giraudeau

Monica Taljaard

Affiliations

Soon will be listed here.

Abstract

Objectives: To describe prevalence of multiple primary outcomes, changes in primary outcomes and target sample sizes between protocols and final reports, and how issues of multiplicity are addressed in pragmatic trials.

Study Design And Setting: Individually randomized trials labeled as pragmatic, published 2014-2019 in MEDLINE and registered with ClinicalTrials.gov.

Results: We identified 262 final reports and located protocols for 159 (61%); primary outcomes were clearly reported in 145 (91%) protocols and 256 (98%) final reports. Thirty (19%) protocols and 38 (15%) final reports had multiple primary outcomes. Primary outcomes were present and identical in 128 (81%) matched protocol-final reports. Among 140 pairs with target sample sizes reported, 28 (20.0%) reduced their target sample size (mean 543 fewer participants per trial) and 16 (11.4%) increased it (mean 192 more participants per trial). Thirteen (29.5%) provided an explanation. Only 2 of 30 (7%) protocols and 4 of 38 (11%) final reports with co-primary outcomes explained how results would be interpreted in light of multiplicity; 21 of 30 (70%) protocols and 20 of 38 (53%) final reports accounted for co-primary outcomes in power calculations.

Conclusion: Co-primary outcomes are common in pragmatic trials; improved transparency around design and analysis decisions involving co-primary outcomes is required.

Citing Articles

Estimating relative risks and risk differences in randomised controlled trials: a systematic review of current practice.

Thompson J, Watson S, Middleton L, Hemming K Trials. 2025; 26(1):1.

PMID: 39748241 PMC: 11694472. DOI: 10.1186/s13063-024-08690-w.

Reporting of and explanations for under-recruitment and over-recruitment in pragmatic trials: a secondary analysis of a database of primary trial reports published from 2014 to 2019.

Nevins P, Nicholls S, Ouyang Y, Carroll K, Hemming K, Weijer C BMJ Open. 2023; 12(12):e067656.

PMID: 36600344 PMC: 9743401. DOI: 10.1136/bmjopen-2022-067656.

Power analysis for cluster randomized trials with continuous coprimary endpoints.

Yang S, Moerbeek M, Taljaard M, Li F Biometrics. 2022; 79(2):1293-1305.

PMID: 35531926 PMC: 11321238. DOI: 10.1111/biom.13692.

References

Khan M, Khan M, Ansari Z, Siddiqi T, Khan S, Riaz I . Prevalence of Multiplicity and Appropriate Adjustments Among Cardiovascular Randomized Clinical Trials Published in Major Medical Journals. JAMA Netw Open. 2020; 3(4):e203082. PMC: 7165301. DOI: 10.1001/jamanetworkopen.2020.3082. View

Sozu T, Sugimoto T, Hamasaki T . Sample size determination in clinical trials with multiple co-primary endpoints including mixed continuous and binary variables. Biom J. 2012; 54(5):716-29. DOI: 10.1002/bimj.201100221. View

Taljaard M, Weijer C, Grimshaw J, Ali A, Brehaut J, Campbell M . Developing a framework for the ethical design and conduct of pragmatic trials in healthcare: a mixed methods research protocol. Trials. 2018; 19(1):525. PMC: 6161426. DOI: 10.1186/s13063-018-2895-x. View

Perlmutter A, Tran V, Dechartres A, Ravaud P . Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials. Ann Oncol. 2016; 28(4):688-695. DOI: 10.1093/annonc/mdw682. View

Hamasaki T, Evans S, Asakura K . Design, data monitoring, and analysis of clinical trials with co-primary endpoints: A review. J Biopharm Stat. 2017; 28(1):28-51. PMC: 6135538. DOI: 10.1080/10543406.2017.1378668. View

Sozu T, Sugimoto T, Hamasaki T . Sample size determination in superiority clinical trials with multiple co-primary correlated endpoints. J Biopharm Stat. 2011; 21(4):650-68. DOI: 10.1080/10543406.2011.551329. View

Taljaard M, McDonald S, Nicholls S, Carroll K, Hey S, Grimshaw J . A search filter to identify pragmatic trials in MEDLINE was highly specific but lacked sensitivity. J Clin Epidemiol. 2020; 124:75-84. DOI: 10.1016/j.jclinepi.2020.05.003. View

Li G, Taljaard M, van den Heuvel E, Levine M, Cook D, Wells G . An introduction to multiplicity issues in clinical trials: the what, why, when and how. Int J Epidemiol. 2016; 46(2):746-755. DOI: 10.1093/ije/dyw320. View

Chen T, Li C, Qin R, Wang Y, Yu D, Dodd J . Comparison of Clinical Trial Changes in Primary Outcome and Reported Intervention Effect Size Between Trial Registration and Publication. JAMA Netw Open. 2019; 2(7):e197242. PMC: 6646984. DOI: 10.1001/jamanetworkopen.2019.7242. View

10.

Moher D, Hopewell S, Schulz K, Montori V, Gotzsche P, Devereaux P . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg. 2011; 10(1):28-55. DOI: 10.1016/j.ijsu.2011.10.001. View

11.

Patsopoulos N . A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011; 13(2):217-24. PMC: 3181997. View

12.

Liang F, Guo X, Zhang S, Xue H, Chen Q, Hu X . Comparison of primary endpoints between publications, registries, and protocols of phase III cancer clinical trials. Oncotarget. 2017; 8(57):97648-97656. PMC: 5722592. DOI: 10.18632/oncotarget.21459. View

13.

Vickerstaff V, Ambler G, King M, Nazareth I, Omar R . Are multiple primary outcomes analysed appropriately in randomised controlled trials? A review. Contemp Clin Trials. 2015; 45(Pt A):8-12. DOI: 10.1016/j.cct.2015.07.016. View

14.

Chan A, Hrobjartsson A, Haahr M, Gotzsche P, Altman D . Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004; 291(20):2457-65. DOI: 10.1001/jama.291.20.2457. View

15.

Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe K, Zwarenstein M . The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350:h2147. DOI: 10.1136/bmj.h2147. View

16.

Chan A, Hrobjartsson A, Jorgensen K, Gotzsche P, Altman D . Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008; 337:a2299. PMC: 2600604. DOI: 10.1136/bmj.a2299. View

17.

Schwartz D, Lellouch J . Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967; 20(8):637-48. DOI: 10.1016/0021-9681(67)90041-0. View

18.

Roland M, Torgerson D . Understanding controlled trials: what outcomes should be measured?. BMJ. 1998; 317(7165):1075. PMC: 1114071. DOI: 10.1136/bmj.317.7165.1075. View

19.

Vanderhout S, Fergusson D, Cook J, Taljaard M . Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis. PLoS Med. 2022; 19(2):e1003896. PMC: 8824332. DOI: 10.1371/journal.pmed.1003896. View

20.

Copsey B, Thompson J, Vadher K, Ali U, Dutton S, Fitzpatrick R . Sample size calculations are poorly conducted and reported in many randomized trials of hip and knee osteoarthritis: results of a systematic review. J Clin Epidemiol. 2018; 104:52-61. DOI: 10.1016/j.jclinepi.2018.08.013. View