Language of Publication Restrictions in Systematic Reviews Gave Different Results Depending on Whether the Intervention Was Conventional or Complementary

Objective: To assess whether language of publication restrictions impact the estimates of an intervention's effectiveness, whether such impact is similar for conventional medicine and complementary medicine interventions, and whether the results are influenced by publication bias and statistical heterogeneity.

Study Design And Setting: We set out to examine the extent to which including reports of randomized controlled trials (RCTs) in languages other than English (LOE) influences the results of systematic reviews, using a broad dataset of 42 language-inclusive systematic reviews, involving 662 RCTs, including both conventional medicine (CM) and complementary and alternative medicine (CAM) interventions.

Results: For CM interventions, language-restricted systematic reviews, compared with language-inclusive ones, did not introduce biased results, in terms of estimates of intervention effectiveness (random effects ration of odds rations ROR=1.02; 95% CI=0.83-1.26). For CAM interventions, however, language-restricted systematic reviews resulted in a 63% smaller protective effect estimate than language-inclusive reviews (random effects ROR=1.63; 95% CI=1.03-2.60).

Conclusion: Language restrictions do not change the results of CM systematic reviews but do substantially alter the results of CAM systematic reviews. These findings are robust even after sensitivity analyses, and do not appear to be influenced by statistical heterogeneity and publication bias.