Development and Evaluation of CAHPS Questions to Assess the Impact of Health Information Technology on Patient Experiences with Ambulatory Care

Background: Little is known about whether health information technology (HIT) affects patient experiences with health care.

Objective: To develop HIT questions that assess patients care experiences not evaluated by existing ambulatory Consumer Assessment of Health Plans and Systems (CAHPS) measures.

Research Design: We reviewed published articles and conducted focus groups and cognitive testing to develop survey questions. We collected data, using mail and the internet, from patients of 69 physicians receiving care at an academic medical center and 2 regional integrated delivery systems in late 2009 and 2010. We evaluated questions and scales about HIT using factor analysis, item-scale correlations, and reliability (internal consistency and physician-level) estimates.

Results: We found support for 3 HIT composites: doctor use of computer (2 items), e-mail (2 items), and helpfulness of provider's website (4 items). Corrected item-scale correlations were 0.37 for the 2 doctor use of computer items and 0.71 for the 2 e-mail items, and ranged from 0.50 to 0.60 for the provider's website items. Cronbach α was high for e-mail (0.83) and provider's website (0.75), but only 0.54 for doctor use of computer. As few as 50 responses per physician would yield reliability of 0.70 for e-mail and provider's website. Two HIT composites, doctor use of computer (P<0.001) and provider's website (P=0.02), were independent predictors of overall ratings of doctors.

Conclusions: New CAHPS HIT items were identified that measure aspects of patient experiences not assessed by the CAHPS C&G 1.0 survey.