Cochrane Systematic Reviews on Traditional Chinese Medicine: What Matters-the Quantity or Quality of Evidence?

Background: Systematic reviews on traditional Chinese medicine (TCM) are constantly increasing. However, if these reviews are to be of practical value, the evidence needs to be relevant, valid, and adequately reported. Cochrane Systematic Reviews (CSRs) are considered as high-quality systematic reviews that can inform health care decision making. Our aim was to provide an overview of the scope, findings, quality and impact of CSRs on the benefits and harms associated with TCM interventions for the treatment and prevention of disease to provide new information for clinical practice and future research.

Methods: The Cochrane Database of Systematic Reviews was searched up to May 2021, and descriptive characteristics were extracted. The correspondence between the questions asked in the CSRs and the available evidence, conclusions and certainty of findings (according to GRADE assessment), methodological quality (AMSTAR 2), and impact (Altmetric Attention Score [AAS], total citations by guideline, and total citations in Web of Science [WoS]) of CSRs were extracted. Tabular and graphical summaries of these descriptive characteristics were constructed.

Results: Of 104 CSRs on TCM identified, 70 diseases belonged to 16 disease systems and contained 1642 primary studies with 157,943 participants. Interventions included Chinese herbal medicine (n = 70), acupuncture (n = 28), TCM exercises (n = 4), and moxibustion (n = 2). Among 1642 primary studies, 662 studies included an intervention group treated with at least one TCM therapy and 980 studies included a combination of therapies. Promising outcomes from the 104 CSRs were divided into endpoint outcomes (34 diseases), doctor- or patient-reported outcomes (27 diseases), and surrogate outcomes (37 diseases). Despite the presence of promising outcomes, only 5/104 CSRs drew overall positive conclusions, 42 CSRs concluded the evidence was insufficient, and 54 failed to draw firm conclusions. GRADE assessments were reported in 41.3% of the CSRs, and the ratings were mostly low or very low. Comparing the questions asked and results obtained, there was frequently a lack of information about specific outcomes. Only 16 CSRs obtained results for all outcomes listed in the methods section. According to AMSTAR 2, 51 CSRs (49.0%) were of low quality. The total number of citations in the WoS was 2135 (mean ± SD: 20.8 ± 21.2), and 38.5% of the CSRs had been cited in guidelines 95 times.

Conclusion: Although TCM is commonly used, evidence of its effectiveness remains largely inconclusive. Rigorous high-quality trials are needed to support the performance of high-quality reviews and to increase the evidence base. It is critical to emphasize quality over quantity in future TCM research.