Evaluating the Evaluators - Developing Evidence of Quality Oversight Effectiveness for Clinical Trial Monitoring: Source Data Verification, Source Data Review, Statistical Monitoring, Key Risk Indicators, and Direct Measure of High Risk Errors

Background: Detailed information about clinical trial quality typically is not published in primary publications. This is a significant issue for physicians, advisory groups, and regulators. In addition, the efficacy of methods to find critical errors that affect trial integrity has not been tested. For more than 20 years, visits to research sites for source data verification (SDV) have been the gold standard method for oversight by pharmaceutical, biotech, device, and vaccine manufacturers. However, there is no evidence of its effectiveness and significant evidence of its lack of effectiveness. GAPS: We consulted documents from the Food and Drug Administration (FDA), European Medicines Association (EMA), and International Council for Harmonization (ICH) to break down components of this pivotal issue into gaps in monitoring, oversight cost, and test validation and effectiveness.

Recommendations: We recommend that the pharma, biotech, academic, and publishing communities develop and provide reference datasets with defined errors for trials similar to the reference standards for analytic tests; systematically test the different monitoring methods using the reference datasets as well as provide sensitivity, specificity, positive and negative predictive values, and time to critical finding for each methodology for different types of trial errors; and publish the details of trial quality in each publication, including the number of major deviations.

Conclusions: This will allow the scientific community to improve measurements of research quality, evaluate new methods, and compare the effectiveness of different oversight approaches in the same environment.