Low Awareness of the Transitivity Assumption in Complex Networks of Interventions: a Systematic Survey from 721 Network Meta-analyses

Background: The transitivity assumption is the cornerstone of network meta-analysis (NMA). Violating transitivity compromises the credibility of the indirect estimates and, by extent, the estimated treatment effects of the comparisons in the network. The present study offers comprehensive empirical evidence on the completeness of reporting and evaluating transitivity in systematic reviews with multiple interventions.

Methods: We screened the datasets of two previous empirical studies, resulting in 361 systematic reviews with NMA published between January 2011 and April 2015. We updated our evidence base with an additional 360 systematic reviews with NMA published between 2016 and 2021, employing a pragmatic approach. We devised assessment criteria for reporting and evaluating transitivity using relevant methodological literature and compared their reporting frequency before and after the PRISMA-NMA statement.

Results: Systematic reviews published after PRISMA-NMA were more likely to provide a protocol (odds ratio (OR): 3.94, 95% CI: 2.79-5.64), pre-plan the transitivity evaluation (OR: 3.01, 95% CI: 1.54-6.23), and report the evaluation and results (OR: 2.10, 95% CI: 1.55-2.86) than those before PRISMA-NMA. However, systematic reviews after PRISMA-NMA were less likely to define transitivity (OR: 0.57, 95% CI: 0.42-0.79) and discuss the implications of transitivity (OR: 0.48, 95% CI: 0.27-0.85) than those published before PRISMA-NMA. Most systematic reviews evaluated transitivity statistically than conceptually (40% versus 12% before PRISMA-NMA, and 54% versus 11% after PRISMA-NMA), with consistency evaluation being the most preferred (34% before versus 47% after PRISMA-NMA). One in five reviews inferred the plausibility of the transitivity (22% before versus 18% after PRISMA-NMA), followed by 11% of reviews that found it difficult to judge transitivity due to insufficient data. In justifying their conclusions, reviews considered mostly the comparability of the trials (24% before versus 30% after PRISMA-NMA), followed by the consistency evaluation (23% before versus 16% after PRISMA-NMA).

Conclusions: Overall, there has been a slight improvement in reporting and evaluating transitivity since releasing PRISMA-NMA, particularly in items related to the systematic review report. Nevertheless, there has been limited attention to pre-planning the transitivity evaluation and low awareness of the conceptual evaluation methods that align with the nature of the assumption.