Tailored Meta-analysis: an Investigation of the Correlation Between the Test Positive Rate and Prevalence

Overview

Journal J Clin Epidemiol

Publisher Elsevier

Specialty Public Health

Date 2018 Oct 3

PMID 30278213

Citations 5

Authors

Brian H Willis

Dyuti Coomar

Mohammed Baragilly

Affiliations

Soon will be listed here.

Abstract

Background And Objective: Meta-analysis may produce estimates that are unrepresentative of a test's performance in practice. Tailored meta-analysis (TMA) circumvents this by deriving an applicable region for the practice and selecting the studies compatible with the region. It requires the test positive rate, r and prevalence, p being estimated for the setting but previous studies have assumed their independence. The aim is to investigate the effects a correlation between r and p has on estimating the applicable region and how this affects TMA.

Methods: Six methods for estimating 99% confidence intervals (CI) for r and p were investigated: Wilson's ± Bonferroni correction, Clopper-Pearson's ± Bonferroni correction, and Hotelling's T statistic ± continuity correction. These were analyzed in terms of the coverage probability using simulation trials over different correlations, sample sizes, and values for r and p. The methods were then applied to two published meta-analyses with associated practice data, and the effects on the applicable region, studies selected, and summary estimates were evaluated.

Results: Hotelling's T statistic with a continuity correction had the highest median coverage (0.9971). This and the Clopper-Pearson method with a Bonferroni correction both had coverage consistently above 0.99. The coverage of Hotelling's CI's varied the least across different correlations. For both meta-analyses, the number of studies selected was largest when Hotelling's T statistic was used to derive the applicable region. In one instance, this increased the sensitivity by over 4% compared with TMA estimates using other methods.

Conclusion: TMA returns estimates that are tailored to practice providing the applicable region is accurately defined. This is most likely when the CI for r and p are estimated using Hotelling's T statistic with a continuity correction. Potentially, the applicable region may be obtained using routine electronic health data.

Citing Articles

Deaths and cardiopulmonary events following colorectal cancer screening-A systematic review with meta-analyses.

Martiny F, Bie A, Jauernik C, Rahbek O, Nielsen S, Gram E PLoS One. 2024; 19(3):e0295900.

PMID: 38483910 PMC: 10939197. DOI: 10.1371/journal.pone.0295900.

Comparing outcomes from tailored meta-analysis with outcomes from a setting specific test accuracy study using routine data of faecal calprotectin testing for inflammatory bowel disease.

Freeman K, Willis B, Ryan R, Taylor-Phillips S, Clarke A BMC Med Res Methodol. 2022; 22(1):192.

PMID: 35820893 PMC: 9275166. DOI: 10.1186/s12874-022-01668-9.

On estimating a constrained bivariate random effects model for meta-analysis of test accuracy studies.

Baragilly M, Willis B Stat Methods Med Res. 2022; 31(2):287-299.

PMID: 34994667 PMC: 8829734. DOI: 10.1177/09622802211065157.

Comparison of Centor and McIsaac scores in primary care: a meta-analysis over multiple thresholds.

Willis B, Coomar D, Baragilly M Br J Gen Pract. 2020; 70(693):e245-e254.

PMID: 32152041 PMC: 7065683. DOI: 10.3399/bjgp20X708833.

Maximum likelihood estimation based on Newton-Raphson iteration for the bivariate random effects model in test accuracy meta-analysis.

Willis B, Baragilly M, Coomar D Stat Methods Med Res. 2019; 29(4):1197-1211.

PMID: 31184270 PMC: 7221455. DOI: 10.1177/0962280219853602.

References

Harbord R, Deeks J, Egger M, Whiting P, Sterne J . A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2006; 8(2):239-51. DOI: 10.1093/biostatistics/kxl004. View

Harbord R, Whiting P, Sterne J, Egger M, Deeks J, Shang A . An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. J Clin Epidemiol. 2009; 61(11):1095-103. DOI: 10.1016/j.jclinepi.2007.09.013. View

Willis B, Hyde C . What is the test's accuracy in my practice population? Tailored meta-analysis provides a plausible estimate. J Clin Epidemiol. 2014; 68(8):847-54. DOI: 10.1016/j.jclinepi.2014.10.002. View

Wynants L, Riley R, Timmerman D, Van Calster B . Random-effects meta-analysis of the clinical utility of tests and prediction models. Stat Med. 2018; 37(12):2034-2052. DOI: 10.1002/sim.7653. View

Willis B . Empirical evidence that disease prevalence may affect the performance of diagnostic tests with an implicit threshold: a cross-sectional study. BMJ Open. 2012; 2(1):e000746. PMC: 3274715. DOI: 10.1136/bmjopen-2011-000746. View

Ransohoff D, Feinstein A . Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978; 299(17):926-30. DOI: 10.1056/NEJM197810262991705. View

Reitsma J, Glas A, Rutjes A, Scholten R, Bossuyt P, Zwinderman A . Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005; 58(10):982-90. DOI: 10.1016/j.jclinepi.2005.02.022. View

Riley R, Ahmed I, Debray T, Willis B, Noordzij J, Higgins J . Summarising and validating test accuracy results across multiple studies for use in clinical practice. Stat Med. 2015; 34(13):2081-103. PMC: 4973708. DOI: 10.1002/sim.6471. View

Willis B, Hyde C . Estimating a test's accuracy using tailored meta-analysis-How setting-specific data may aid study selection. J Clin Epidemiol. 2014; 67(5):538-46. DOI: 10.1016/j.jclinepi.2013.10.016. View

10.

Chu H, Cole S . Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006; 59(12):1331-2. DOI: 10.1016/j.jclinepi.2006.06.011. View

11.

Rutter C, Gatsonis C . A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001; 20(19):2865-84. DOI: 10.1002/sim.942. View

12.

Koran L . The reliability of clinical methods, data and judgments (second of two parts). N Engl J Med. 1975; 293(14):695-701. DOI: 10.1056/NEJM197510022931405. View

13.

Leeflang M, Deeks J, Rutjes A, Reitsma J, Bossuyt P . Bivariate meta-analysis of predictive values of diagnostic tests can be an alternative to bivariate meta-analysis of sensitivity and specificity. J Clin Epidemiol. 2012; 65(10):1088-97. DOI: 10.1016/j.jclinepi.2012.03.006. View

14.

Willis B . Spectrum bias--why clinicians need to be cautious when applying diagnostic test studies. Fam Pract. 2008; 25(5):390-6. DOI: 10.1093/fampra/cmn051. View

15.

Willis B, Riley R . Measuring the statistical validity of summary meta-analysis and meta-regression results for use in clinical practice. Stat Med. 2017; 36(21):3283-3301. PMC: 5575530. DOI: 10.1002/sim.7372. View

16.

Chu H, Nie L, Cole S, Poole C . Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative parameterizations and model selection. Stat Med. 2009; 28(18):2384-99. DOI: 10.1002/sim.3627. View

17.

Leeflang M, Bossuyt P, Irwig L . Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol. 2008; 62(1):5-12. DOI: 10.1016/j.jclinepi.2008.04.007. View