Development of a Congenital Heart Surgery Composite Quality Metric: Part 2-Analytic Methods

Background: We describe the statistical methods and results related to development of the first congenital heart surgery composite quality measure.

Methods: The composite measure was developed using The Society of Thoracic Surgeons Congenital Heart Surgery Database (2012 to 2015), Bayesian hierarchical modeling, and the current Society of Thoracic Surgeons risk model for case-mix adjustment. It consists of a mortality domain (operative mortality) and morbidity domain (major complications and postoperative length of stay). We evaluated several potential weighting schemes and properties of the final composite measure, including reliability (signal-to-noise ratio) and hospital classification in various performance categories.

Results: Overall, 100 hospitals (78,425 operations) were included. Each adjusted metric included in the composite varied across hospitals: operative mortality (median, 3.1%; 10th to 90th percentile, 2.1% to 4.4%) major complications (median 11.7%, 10th to 90th percentile, 6.4% to 17.4%), and length of stay (median, 7.0 days; 10th to 90th percentile, 5.9 to 8.2 days). In the final composite weighting scheme selected, mortality had the greatest influence, followed by major complications and length of stay (correlation with overall composite score of 0.87, 0.69, and 0.47, respectively). Reliability of the composite measure was 0.73 compared with 0.59 for mortality alone. The distribution of hospitals across composite measure performance categories (defined by whether the 95% credible interval overlapped The Society of Thoracic Surgeons average) was 75% (same as expected), 9% (worse than expected), and 16% (better than expected).

Conclusions: This congenital heart surgery composite measure incorporates aspects of both morbidity and mortality, has clinical face validity, and greater ability to discriminate hospital performance compared with mortality alone. Ongoing efforts will support the use of the composite measure in benchmarking and quality improvement activities.