Risk-Adjusted Variation of Publicly Reported Emergency Department Timeliness Measures

Study Objective: The Centers for Medicare & Medicaid Services (CMS) recently published emergency department (ED) timeliness measures. These data show substantial variation in hospital performance and suggest the need for process improvement initiatives. However, the CMS measures are not risk adjusted and may provide misleading information about hospital performance and variation. We hypothesize that substantial hospital-level variation will persist after risk adjustment.

Methods: This cross-sectional study included hospitals that participated in the Emergency Department Benchmarking Alliance and CMS ED measure reporting in 2012. Outcomes included the CMS measures corresponding to median annual boarding time, length of stay of admitted patients, length of stay of discharged patients, and waiting time of discharged patients. Covariates included hospital structural characteristics and case-mix information from the American Hospital Association Survey, CMS cost reports, and the Emergency Department Benchmarking Alliance. We used a γ regression with a log link to model the skewed outcomes. We used indirect standardization to create risk-adjusted measures. We defined "substantial" variation as coefficient of variation greater than 0.15.

Results: The study cohort included 723 hospitals. Risk-adjusted performance on the CMS measures varied substantially across hospitals, with coefficient of variation greater than 0.15 for all measures. Ratios between the 10th and 90th percentiles of performance ranged from 1.5-fold for length of stay of discharged patients to 3-fold for waiting time of discharged patients.

Conclusion: Policy-relevant variations in publicly reported CMS ED timeliness measures persist after risk adjustment for nonmodifiable hospital and case-mix characteristics. Future "positive deviance" studies should identify modifiable process measures associated with high performance.