Towards Usable E-Health. A Systematic Review of Usability Questionnaires

Overview

Journal Appl Clin Inform

Publisher Thieme

Specialties Health Services
Medical Informatics

Date 2017 May 11

PMID 28487932

Citations 39

Authors

Vanessa E C Sousa

Karen Dunn Lopez

Affiliations

Soon will be listed here.

Abstract

Background: The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.

Objectives: To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.

Methods: We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.

Results: Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).

Conclusions: Existing questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.

Citing Articles

German Version of the Telehealth Usability Questionnaire and Derived Short Questionnaires for Usability and Perceived Usefulness in Health Care Assessment in Telehealth and Digital Therapeutics: Instrument Validation Study.

Zimmermann J, Morf H, Scharf F, Knitza J, Moeller H, Muehlensiepen F JMIR Hum Factors. 2024; 11:e57771.

PMID: 39571151 PMC: 11621722. DOI: 10.2196/57771.

Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy.

Torring M, Logacjov A, Braendvik S, Ustad A, Roeleveld K, Bardal E PLoS One. 2024; 19(9):e0308853.

PMID: 39312531 PMC: 11419372. DOI: 10.1371/journal.pone.0308853.

Usability testing of the Set Brave Goals app, a continence goal-selection app for children with spina bifida.

Szymanski K, Carroll A, Bennet Jr W, Misseri R J Pediatr Urol. 2024; 20(4):724.e1-724.e7.

PMID: 38969555 PMC: 11330350. DOI: 10.1016/j.jpurol.2024.06.025.

Utility and usability of a wearable system and progressive-challenge cued exercise program for encouraging use of the more involved arm at-home after stroke-a feasibility study with case reports.

Horder J, Mrotek L, Casadio M, Bassindale K, McGuire J, Scheidt R J Neuroeng Rehabil. 2024; 21(1):66.

PMID: 38685012 PMC: 11059679. DOI: 10.1186/s12984-024-01359-0.

A Physical Activity and Diet Just-in-Time Adaptive Intervention to Reduce Blood Pressure: The myBPmyLife Study Rationale and Design.

Golbus J, Jeganathan V, Stevens R, Ekechukwu W, Farhan Z, Contreras R J Am Heart Assoc. 2024; 13(2):e031234.

PMID: 38226507 PMC: 10926831. DOI: 10.1161/JAHA.123.031234.

References

Ratwani R, Hettinger A, Fairbanks R . Barriers to comparing the usability of electronic health records. J Am Med Inform Assoc. 2016; 24(e1):e191-e193. PMC: 7651928. DOI: 10.1093/jamia/ocw117. View

Cox C, Wysham N, Walton B, Jones D, Cass B, Tobin M . Development and usability testing of a Web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care. 2015; 5:6. PMC: 4385299. DOI: 10.1186/s13613-015-0045-0. View

Lee T, Mills M, Bausell B, Lu M . Two-stage evaluation of the impact of a nursing information system in Taiwan. Int J Med Inform. 2008; 77(10):698-707. DOI: 10.1016/j.ijmedinf.2008.03.004. View

Lim M, Hocking J, Aitken C, Fairley C, Jordan L, Lewis J . Impact of text and email messaging on the sexual health of young people: a randomised controlled trial. J Epidemiol Community Health. 2011; 66(1):69-74. DOI: 10.1136/jech.2009.100396. View

Johnson E, Nelson C . Values and pitfalls of the use of administrative databases for outcomes assessment. J Urol. 2013; 190(1):17-8. PMC: 4114235. DOI: 10.1016/j.juro.2013.04.048. View

Friedman K, Noyes J, Parkin C . 2-Year follow-up to STeP trial shows sustainability of structured self-monitoring of blood glucose utilization: results from the STeP practice logistics and usability survey (STeP PLUS). Diabetes Technol Ther. 2013; 15(4):344-7. DOI: 10.1089/dia.2012.0304. View

Heikkinen K, Suomi R, Jaaskelainen M, Kaljonen A, Leino-Kilpi H, Salantera S . The creation and evaluation of an ambulatory orthopedic surgical patient education web site to support empowerment. Comput Inform Nurs. 2010; 28(5):282-90. DOI: 10.1097/NCN.0b013e3181ec23e6. View

Lin M, Hong R, Chang J, Ke X . Usage Position and Virtual Keyboard Design Affect Upper-Body Kinematics, Discomfort, and Usability during Prolonged Tablet Typing. PLoS One. 2015; 10(12):e0143585. PMC: 4667996. DOI: 10.1371/journal.pone.0143585. View

Streiner D . Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003; 80(1):99-103. DOI: 10.1207/S15327752JPA8001_18. View

10.

Staggers N, Kobus D, Brown C . Nurses' evaluations of a novel design for an electronic medication administration record. Comput Inform Nurs. 2007; 25(2):67-75. DOI: 10.1097/01.NCN.0000263981.38801.be. View

11.

Kim M, Shapiro J, Genes N, Aguilar M, Mohrer D, Baumlin K . A pilot study on usability analysis of emergency department information system by nurses. Appl Clin Inform. 2013; 3(1):135-53. PMC: 3613014. DOI: 10.4338/ACI-2011-11-RA-0065. View

12.

Joshi A, Amadi C, Trout K, Obaro S . Evaluation of an interactive surveillance system for monitoring acute bacterial infections in Nigeria. Perspect Health Inf Manag. 2014; 11:1f. PMC: 3995493. View

13.

Yui B, Jim W, Chen M, Hsu J, Liu C, Lee T . Evaluation of computerized physician order entry system-a satisfaction survey in Taiwan. J Med Syst. 2012; 36(6):3817-24. DOI: 10.1007/s10916-012-9854-y. View

14.

Ozel D, Bilge U, Zayim N, Cengiz M . A web-based intensive care clinical decision support system: from design to evaluation. Inform Health Soc Care. 2012; 38(2):79-92. DOI: 10.3109/17538157.2012.710687. View

15.

Moattari M, Moosavinasab E, Dabbaghmanesh M, Zarifsanaiey N . Validating a Web-based Diabetes Education Program in continuing nursing education: knowledge and competency change and user perceptions on usability and quality. J Diabetes Metab Disord. 2015; 13:70. PMC: 4470346. DOI: 10.1186/2251-6581-13-70. View

16.

Freire L, Arezes P, Campos J . A literature review about usability evaluation methods for e-learning platforms. Work. 2012; 41 Suppl 1:1038-44. DOI: 10.3233/WOR-2012-0281-1038. View

17.

Lilholt P, Jensen M, Hejlesen O . Heuristic evaluation of a telehealth system from the Danish TeleCare North Trial. Int J Med Inform. 2015; 84(5):319-26. DOI: 10.1016/j.ijmedinf.2015.01.012. View

18.

Shyr C, Kushniruk A, Wasserman W . Usability study of clinical exome analysis software: top lessons learned and recommendations. J Biomed Inform. 2014; 51:129-36. DOI: 10.1016/j.jbi.2014.05.004. View

19.

Wilkinson A, Forbes A, Bloomfield J, Fincham Gee C . An exploration of four web-based open and flexible learning modules in post-registration nurse education. Int J Nurs Stud. 2004; 41(4):411-24. DOI: 10.1016/j.ijnurstu.2003.11.001. View

20.

Sawka A, Straus S, Gafni A, Meiyappan S, OBrien M, Brierley J . A usability study of a computerized decision aid to help patients with, early stage papillary thyroid carcinoma in, decision-making on adjuvant radioactive iodine treatment. Patient Educ Couns. 2010; 84(2):e24-7. DOI: 10.1016/j.pec.2010.07.038. View