Quantifying Risk Stratification Provided by Diagnostic Tests and Risk Predictions: Comparison to AUC and Decision Curve Analysis

A property of diagnostic tests and risk models deserving more attention is risk stratification, defined as the ability of a test or model to separate those at high absolute risk of disease from those at low absolute risk. Risk stratification fills a gap between measures of classification (ie, area under the curve (AUC)) that do not require absolute risks and decision analysis that requires not only absolute risks but also subjective specification of costs and utilities. We introduce mean risk stratification (MRS) as the average change in risk of disease (posttest-pretest) revealed by a diagnostic test or risk model dichotomized at a risk threshold. Mean risk stratification is particularly valuable for rare conditions, where AUC can be high but MRS can be low, identifying situations that temper overenthusiasm for screening with the new test/model. We apply MRS to the controversy over who should get testing for mutations in BRCA1/2 that cause high risks of breast and ovarian cancers. To reveal different properties of risk thresholds to refer women for BRCA1/2 testing, we propose an eclectic approach considering MRS and other metrics. The value of MRS is to interpret AUC in the context of BRCA1/2 mutation prevalence, providing a range of risk thresholds at which a risk model is "optimally informative," and to provide insight into why net benefit arrives to its conclusion.