The Fragility of Statistically Significant Binary Outcomes for Treating Achilles Tendinopathy: A Systematic Review of Randomized Trials

Overview

Journal Foot Ankle Orthop

Publisher Sage Publications

Date 2024 Nov 22

PMID 39575398

Authors

Omkar S Anaspure

Shiv Patel

Anthony N Baumann

Andrew Newsom

Albert T Anastasio

Annunziato Amendola

Affiliations

Soon will be listed here.

Abstract

Background: Randomized controlled trials (RCTs) are the gold standard for treatment efficacy, but foot and ankle RCTs are often small or inconsistent. The Fragility Index (FI) evaluates the stability of significant findings. This study assessed the fragility of RCT outcomes for Achilles tendon pathology (ATP) interventions.

Methods: This systematic review queried PubMed up to May 14, 2024, for RCTs on ATP interventions. RCTs with significant binary outcomes were included. Two reviewers assessed eligibility, extracted data, calculated FIs, and evaluated risk of bias. Frequency-weighted means were used for narrative synthesis.

Results: Eleven RCTs with 4506 patients (mean cohort size: 409.64 ± 160.54) and a mean age of 36.97 ± 13.51 years (n = 4356; 96.67%) were included, covering 24 binary outcomes. The median FI across all outcomes was 3 (interquartile range 1-4; mean 3.92), indicating that changing the outcome of just a few patients could shift a study's results from statistically significant to nonsignificant. Trials having an FI ≤3 comprised 58.33%. Three outcomes (12.5%) had an FI of zero after recalculating values using the two-sided Fisher exact test. Half of the outcomes were robust. No RCT reported FIs or adjusted significance for multiple testing. Most studies (81.82%) performed 2 or more statistical tests, with an average of 30.81 ± 41.28 values reported per study. The overall risk of bias was low in 1 study (9.09%) and moderate in 7 (63.64%). Most studies had low risk of bias in randomization (72.73%) and missing outcome data (90.91%).

Conclusion: The FI assesses the fragility of statistically significant binary results, revealing that many ATP RCTs have fragile outcomes due to small sample sizes. A median FI of 3 means that changing the outcome of 3 patients could shift a study's results from statistically significant to nonsignificant.

References

Montori V, Devereaux P, Adhikari N, Burns K, Eggert C, Briel M . Randomized trials stopped early for benefit: a systematic review. JAMA. 2005; 294(17):2203-9. DOI: 10.1001/jama.294.17.2203. View

Mafi N, Lorentzon R, Alfredson H . Superior short-term results with eccentric calf muscle training compared to concentric training in a randomized prospective multicenter study on patients with chronic Achilles tendinosis. Knee Surg Sports Traumatol Arthrosc. 2001; 9(1):42-7. DOI: 10.1007/s001670000148. View

Herndon C, McCormick K, Gazgalis A, Bixby E, Levitsky M, Neuwirth A . Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review. Arthroplast Today. 2021; 11:239-251. PMC: 8517286. DOI: 10.1016/j.artd.2021.08.018. View

Matar H, Platt S . Overview of randomised controlled trials in orthopaedic research: search for significant findings. Eur J Orthop Surg Traumatol. 2019; 29(6):1163-1168. DOI: 10.1007/s00590-019-02436-0. View

Bassler D, Briel M, Montori V, Lane M, Glasziou P, Zhou Q . Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010; 303(12):1180-7. DOI: 10.1001/jama.2010.310. View

Huttunen T, Kannus P, Rolf C, Fellander-Tsai L, Mattila V . Acute achilles tendon ruptures: incidence of injury and surgery in Sweden between 2001 and 2012. Am J Sports Med. 2014; 42(10):2419-23. DOI: 10.1177/0363546514540599. View

Sussmilch-Leitch S, Collins N, Bialocerkowski A, Warden S, Crossley K . Physical therapies for Achilles tendinopathy: systematic review and meta-analysis. J Foot Ankle Res. 2012; 5(1):15. PMC: 3537637. DOI: 10.1186/1757-1146-5-15. View

Armstrong R . When to use the Bonferroni correction. Ophthalmic Physiol Opt. 2014; 34(5):502-8. DOI: 10.1111/opo.12131. View

Dettori J, Norvell D . How Fragile Are the Results of a Trial? The Fragility Index. Global Spine J. 2020; 10(7):940-942. PMC: 7485073. DOI: 10.1177/2192568220941684. View

10.

Myhrvold S, Brouwer E, Andresen T, Rydevik K, Amundsen M, Grun W . Nonoperative or Surgical Treatment of Acute Achilles' Tendon Rupture. N Engl J Med. 2022; 386(15):1409-1420. DOI: 10.1056/NEJMoa2108447. View

11.

Keene D, Alsousou J, Harrison P, Hulley P, Wagland S, Parsons S . Platelet rich plasma injection for acute Achilles tendon rupture: PATH-2 randomised, placebo controlled, superiority trial. BMJ. 2019; 367:l6132. PMC: 6863552. DOI: 10.1136/bmj.l6132. View

12.

Go C, Maldonado D, Go B, Annin S, Saks B, Jimenez A . The Fragility Index of Total Hip Arthroplasty Randomized Control Trials: A Systematic Review. J Am Acad Orthop Surg. 2022; 30(9):e741-e750. DOI: 10.5435/JAAOS-D-21-00489. View

13.

Olsson N, Silbernagel K, Eriksson B, Sansone M, Brorsson A, Nilsson-Helander K . Stable surgical repair with accelerated rehabilitation versus nonsurgical treatment for acute Achilles tendon ruptures: a randomized controlled study. Am J Sports Med. 2013; 41(12):2867-76. DOI: 10.1177/0363546513503282. View

14.

Fackler N, Karasavvidis T, Ehlers C, Callan K, Lai W, Parisien R . The Statistical Fragility of Operative vs Nonoperative Management for Achilles Tendon Rupture: A Systematic Review of Comparative Studies. Foot Ankle Int. 2022; 43(10):1331-1339. PMC: 9527367. DOI: 10.1177/10711007221108078. View

15.

Kuhn J, Greenfield M, Wojtys E . A statistics primer. Statistical tests for discrete data. Am J Sports Med. 1997; 25(4):585-6. DOI: 10.1177/036354659702500425. View

16.

Paoloni J, Appleyard R, Nelson J, Murrell G . Topical glyceryl trinitrate treatment of chronic noninsertional achilles tendinopathy. A randomized, double-blind, placebo-controlled trial. J Bone Joint Surg Am. 2004; 86(5):916-22. DOI: 10.2106/00004623-200405000-00005. View

17.

Silbernagel K, Thomee R, Thomee P, Karlsson J . Eccentric overload training for patients with chronic Achilles tendon pain--a randomised controlled study with reliability testing of the evaluation methods. Scand J Med Sci Sports. 2001; 11(4):197-206. DOI: 10.1034/j.1600-0838.2001.110402.x. View

18.

Ho A . The Fragility Index for Assessing the Robustness of the Statistically Significant Results of Experimental Clinical Studies. J Gen Intern Med. 2021; 37(1):206-211. PMC: 8739402. DOI: 10.1007/s11606-021-06999-9. View

19.

Cote M, Asnis P, Hutchinson I, Berkson E . Editorial Commentary: The Statistical Fragility Index of Medical Trials Is Low By Design: Critical Evaluation of Confidence Intervals Is Required. Arthroscopy. 2024; 40(3):1006-1008. DOI: 10.1016/j.arthro.2023.10.010. View

20.

Lydersen S, Pradhan V, Senchaudhuri P, Laake P . Choice of test for association in small sample unordered r x c tables. Stat Med. 2007; 26(23):4328-43. DOI: 10.1002/sim.2839. View