Effects of Survey Mode, Patient Mix, and Nonresponse on CAHPS Hospital Survey Scores

Overview

Journal Health Serv Res

Specialty Health Services

Date 2009 Mar 26

PMID 19317857

Citations 139

Authors

Marc N Elliott

Alan M Zaslavsky

Elizabeth Goldstein

William Lehrman

Katrin Hambarsoomians

Megan K Beckett

Laura Giordano

Affiliations

Soon will be listed here.

Abstract

Objective: To evaluate the need for survey mode adjustments to hospital care evaluations by discharged inpatients and develop the appropriate adjustments.

Data Source: A total of 7,555 respondents from a 2006 national random sample of 45 hospitals who completed the CAHPS Hospital (HCAHPS [Hospital Consumer Assessments of Healthcare Providers and Systems]) Survey.

Study Design/data Collection/extraction Methods: We estimated mode effects in linear models that predicted each HCAHPS outcome from hospital-fixed effects and patient-mix adjustors.

Principal Findings: Patients randomized to the telephone and active interactive voice response (IVR) modes provided more positive evaluations than patients randomized to mail and mixed (mail with telephone follow-up) modes, with some effects equivalent to more than 30 percentile points in hospital rankings. Mode effects are consistent across hospitals and are generally larger than total patient-mix effects. Patient-mix adjustment accounts for any nonresponse bias that could have been addressed through weighting.

Conclusions: Valid comparisons of hospital performance require that reported hospital scores be adjusted for survey mode and patient mix.

Citing Articles

Nonresponse Bias on Inpatient Rehabilitation Hospitals' Experience of Care Quality Measure Scores.

Heinemann A, Deutsch A, Brewington D, Foster L, Cook K, Miskovic A Arch Rehabil Res Clin Transl. 2025; 6(4):100363.

PMID: 39822198 PMC: 11734007. DOI: 10.1016/j.arrct.2024.100363.

Changes in Patient Care Experience After Private Equity Acquisition of US Hospitals.

Bhatla A, Bartlett V, Liu M, Zheng Z, Wadhera R JAMA. 2025; 333(6):490-497.

PMID: 39786740 PMC: 11815525. DOI: 10.1001/jama.2024.23450.

Factors influencing patient experience in hospital wards: a systematic review.

Guan T, Chen X, Li J, Zhang Y BMC Nurs. 2024; 23(1):527.

PMID: 39090643 PMC: 11295641. DOI: 10.1186/s12912-024-02054-0.

Factors associated with self-reports of limitations in activities of daily living among Medicare Fee-for-Service recipients.

Hays R, Haas A, Haviland A, Martino S, Orr N, Binion J BMC Geriatr. 2024; 24(1):648.

PMID: 39090545 PMC: 11295787. DOI: 10.1186/s12877-024-05242-4.

No Association Between Medicare Advantage Providers' Network Restrictiveness and Star Rating Between 2013 and 2017: An Observational Study.

Dai D, Feyman Y, Figueroa J, Frakt A, Garrido M J Gen Intern Med. 2024; 40(2):412-419.

PMID: 39028405 PMC: 11803060. DOI: 10.1007/s11606-024-08938-w.

References

Burroughs T, Waterman B, Cira J, Desikan R, Dunagan W . Patient satisfaction measurement strategies: a comparison of phone and mail methods. Jt Comm J Qual Improv. 2001; 27(7):349-61. DOI: 10.1016/s1070-3241(01)27030-8. View

Rodriguez H, von Glahn T, Rogers W, Chang H, Fanjiang G, Safran D . Evaluating patients' experiences with individual physicians: a randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Med Care. 2006; 44(2):167-74. DOI: 10.1097/01.mlr.0000196961.00933.8e. View

Zaslavsky A, Zaborski L, Ding L, Shaul J, Cioffi M, Cleary P . Adjusting Performance Measures to Ensure Equitable Plan Comparisons. Health Care Financ Rev. 2014; 22(3):109-126. PMC: 4194711. View

Elliott M, Beckett M, Chong K, Hambarsoomians K, Hays R . How do proxy responses and proxy-assisted responses differ from what Medicare beneficiaries might have reported about their health care?. Health Serv Res. 2008; 43(3):833-48. PMC: 2442242. DOI: 10.1111/j.1475-6773.2007.00820.x. View

Kim M, Zaslavsky A, Cleary P . Adjusting Pediatric Consumer Assessment of Health Plans Study (CAHPS) Scores to Ensure Fair Comparison of Health Plan Performances. Med Care. 2005; 43(1):44-52. View

de Vries H, Elliott M, Hepner K, Keller S, Hays R . Equivalence of mail and telephone responses to the CAHPS Hospital Survey. Health Serv Res. 2005; 40(6 Pt 2):2120-39. PMC: 1361249. DOI: 10.1111/j.1475-6773.2005.00479.x. View

Rubin H . Can patients evaluate the quality of hospital care?. Med Care Rev. 1991; 47(3):267-326. DOI: 10.1177/107755879004700302. View

Hepner K, Brown J, Hays R . Comparison of mail and telephone in assessing patient experiences in receiving care from medical group practices. Eval Health Prof. 2005; 28(4):377-89. DOI: 10.1177/0163278705281074. View

Weech-Maldonado R, Elliott M, Oluwole A, Schiller K, Hays R . Survey response style and differential use of CAHPS rating scales by Hispanics. Med Care. 2008; 46(9):963-8. PMC: 3103722. DOI: 10.1097/MLR.0b013e3181791924. View

10.

Elliott M, SWARTZ R, Adams J, Spritzer K, Hays R . Case-mix adjustment of the National CAHPS benchmarking data 1.0: a violation of model assumptions?. Health Serv Res. 2001; 36(3):555-73. PMC: 1089242. View

11.

Heje H, Vedsted P, Olesen F . A cluster-randomized trial of the significance of a reminder procedure in a patient evaluation survey in general practice. Int J Qual Health Care. 2006; 18(3):232-7. DOI: 10.1093/intqhc/mzl006. View

12.

Elliott M, Edwards C, Angeles J, Hambarsoomians K, Hays R . Patterns of unit and item nonresponse in the CAHPS Hospital Survey. Health Serv Res. 2005; 40(6 Pt 2):2096-119. PMC: 1361246. DOI: 10.1111/j.1475-6773.2005.00476.x. View

13.

Mazor K, Clauser B, Field T, Yood R, Gurwitz J . A demonstration of the impact of response bias on the results of patient satisfaction surveys. Health Serv Res. 2002; 37(5):1403-17. PMC: 1464019. DOI: 10.1111/1475-6773.11194. View

14.

Goldstein E, Farquhar M, Crofton C, Darby C, Garfinkel S . Measuring hospital care from the patients' perspective: an overview of the CAHPS Hospital Survey development process. Health Serv Res. 2005; 40(6 Pt 2):1977-95. PMC: 1361247. DOI: 10.1111/j.1475-6773.2005.00477.x. View

15.

Zaslavsky A, Zaborski L, Cleary P . Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Med Care. 2002; 40(6):485-99. DOI: 10.1097/00005650-200206000-00006. View

16.

Barkley W, Furse D . Changing priorities for improvement: the impact of low response rates in patient satisfaction. Jt Comm J Qual Improv. 1996; 22(6):427-33. DOI: 10.1016/s1070-3241(16)30245-0. View

17.

OMalley A, Zaslavsky A, Elliott M, Zaborski L, Cleary P . Case-mix adjustment of the CAHPS Hospital Survey. Health Serv Res. 2005; 40(6 Pt 2):2162-81. PMC: 1361241. DOI: 10.1111/j.1475-6773.2005.00470.x. View

18.

Fowler Jr F, Gallagher P, Nederend S . Comparing telephone and mail responses to the CAHPS survey instrument. Consumer Assessment of Health Plans Study. Med Care. 1999; 37(3 Suppl):MS41-9. DOI: 10.1097/00005650-199903001-00005. View

19.

Lasek R, Barkley W, Harper D, Rosenthal G . An evaluation of the impact of nonresponse bias on patient satisfaction surveys. Med Care. 1997; 35(6):646-52. DOI: 10.1097/00005650-199706000-00009. View

20.

Etter J, Perneger T, Rougemont A . Does sponsorship matter in patient satisfaction surveys? A randomized trial. Med Care. 1996; 34(4):327-35. DOI: 10.1097/00005650-199604000-00004. View