Feasibility Study and Evaluation of Expert Opinion on the Semi-automated Meta-analysis and the Conventional Meta-analysis

Purpose: To assess the feasibility and acceptance of the semi-automated meta-analysis (SAMA). The objectives are twofold, namely (1) to compare expert opinion on the quality of protocols, methods, and results of one conventional meta-analysis (CMA) and one SAMA and (2) to compare the time to execute the CMA and the SAMA.

Methods: Experts evaluated the protocols and manuscripts/reports of the CMA and SAMA conducted independently on the safety of metronidazole in pregnancy. Expert opinion was collected using AMSTAR 2 checklist. Time spent was recorded using case report forms.

Results: The overall scores of the opinion of all experts for protocols, methods, and results for SAMA (6.75) and CMA (6.87) were not statistically different (p = 0.88). The experts' confidence in the results of each MA was 7.89 ± 1.17 and 8.11 ± 0.92, respectively. The time to completion was 14 working days for SAMA and 24.7 for CMA. MA tasks such as calculation of effect estimates, subgroup/sensitivity analysis, and publication bias investigation required no investment in time for SAMA.

Conclusion: In conclusion, our study demonstrated the feasibility of SAMA and suggests acceptance for risk assessment by an expert committee. Our results suggest that SAMA reduces the time required for a MA without altering expert confidence in the methodological and scientific rigor. As our study was limited to one example, the generalization of our results requires confirmation by other studies.