Improving Perceived and Actual Text Difficulty for Health Information Consumers Using Semi-automated Methods

Overview

Journal AMIA Annu Symp Proc

Specialty Medical Informatics

Date 2013 Jan 11

PMID 23304324

Citations 12

Authors

Gondy Leroy

James E Endicott

Obay Mouradi

David Kauchak

Melissa L Just

Affiliations

Soon will be listed here.

Abstract

We are developing algorithms for semi-automated simplification of medical text. Based on lexical and grammatical corpus analysis, we identified a new metric, term familiarity, to help estimate text difficulty. We developed an algorithm that uses term familiarity to identify difficult text and select easier alternatives from lexical resources such as WordNet, UMLS and Wiktionary. Twelve sentences were simplified to measure perceived difficulty using a 5-point Likert scale. Two documents were simplified to measure actual difficulty by posing questions with and without the text present (information understanding and retention). We conducted a user study by inviting participants (N=84) via Amazon Mechanical Turk. There was a significant effect of simplification on perceived difficulty (p<.001). We also saw slightly improved understanding with better question-answering for simplified documents but the effect was not significant (p=.097). Our results show how term familiarity is a valuable component in simplifying text in an efficient and scalable manner.

Citing Articles

Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1.

Ayre J, Bonner C, Muscat D, Dunn A, Harrison E, Dalmazzo J JMIR Form Res. 2023; 7:e40645.

PMID: 36787164 PMC: 9975914. DOI: 10.2196/40645.

A survey of automated methods for biomedical text simplification.

Ondov B, Attal K, Demner-Fushman D J Am Med Inform Assoc. 2022; 29(11):1976-1988.

PMID: 36083212 PMC: 10161533. DOI: 10.1093/jamia/ocac149.

The Role of Surface, Semantic and Grammatical Features on Simplification of Spanish Medical Texts: A User Study.

Mukherjee P, Leroy G, Kauchak D, Navarrete B, Diaz D, Colina S AMIA Annu Symp Proc. 2018; 2017:1322-1331.

PMID: 29854201 PMC: 5977682.

Health IT Usability Focus Section: Adapting EHR-Based Medication Instructions to Comply with Plain Language Guidance-A Randomized Experiment.

Ancker J, Send A, Hafeez B, Osorio S, Abramson E Appl Clin Inform. 2017; 8(4):1127-1143.

PMID: 29241250 PMC: 5802303. DOI: 10.4338/ACI-2017-06-RA-0111.

Measuring Text Difficulty Using Parse-Tree Frequency.

Kauchak D, Leroy G, Hogue A J Assoc Inf Sci Technol. 2017; 68(9):2088-2100.

PMID: 29057293 PMC: 5644354. DOI: 10.1002/asi.23855.

References

Kalichman S, Benotsch E, Suarez T, Catz S, Miller J, Rompa D . Health literacy and health-related knowledge among persons living with HIV/AIDS. Am J Prev Med. 2000; 18(4):325-31. DOI: 10.1016/s0749-3797(00)00121-5. View

Sprouse J . A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory. Behav Res Methods. 2011; 43(1):155-67. PMC: 3048456. DOI: 10.3758/s13428-010-0039-7. View

Von Wagner C, Semmler C, Good A, Wardle J . Health literacy and self-efficacy for participating in colorectal cancer screening: The role of information processing. Patient Educ Couns. 2009; 75(3):352-7. DOI: 10.1016/j.pec.2009.03.015. View

Kandula S, Curtis D, Zeng-Treitler Q . A semantic and syntactic text simplification tool for health content. AMIA Annu Symp Proc. 2011; 2010:366-70. PMC: 3041424. View

Weiss B, Palmer R . Relationship between health care costs and very low literacy skills in a medically needy and indigent Medicaid population. J Am Board Fam Pract. 2004; 17(1):44-7. DOI: 10.3122/jabfm.17.1.44. View

Baker L, Wagner T, Singer S, Bundorf M . Use of the Internet and e-mail for health care information: results from a national survey. JAMA. 2003; 289(18):2400-6. DOI: 10.1001/jama.289.18.2400. View

Baker D, Williams M, Parker R, Gazmararian J, Nurss J . Development of a brief test to measure functional health literacy. Patient Educ Couns. 2003; 38(1):33-42. DOI: 10.1016/s0738-3991(98)00116-5. View

Davis T, Dolan N, Ferreira M, Tomori C, Green K, Sipler A . The role of inadequate health literacy skills in colorectal cancer screening. Cancer Invest. 2001; 19(2):193-200. DOI: 10.1081/cnv-100000154. View

Leroy G, Endicott J . Term Familiarity to indicate Perceived and Actual Difficulty of Text in Medical Digital Libraries. Digit Libraries Cult Herit Knowl Dissem Future Creat (2011). 2015; 7008:307-310. PMC: 4662562. DOI: 10.1007/978-3-642-24826-9_38. View

10.

Mazor K, Dodd K, Kunches L . Communicating hospital infection data to the public: a study of consumer responses and preferences. Am J Med Qual. 2009; 24(2):108-15. DOI: 10.1177/1062860608330827. View

11.

Janz N, BECKER M . The Health Belief Model: a decade later. Health Educ Q. 1984; 11(1):1-47. DOI: 10.1177/109019818401100101. View

12.

Parker R, Baker D, Williams M, Nurss J . The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995; 10(10):537-41. DOI: 10.1007/BF02640361. View

13.

Kandula N, Nsiah-Kumi P, Makoul G, Sager J, Zei C, Glass S . The relationship between health literacy and knowledge improvement after a multimedia type 2 diabetes education program. Patient Educ Couns. 2009; 75(3):321-7. DOI: 10.1016/j.pec.2009.04.001. View

14.

Elhadad N . Comprehending technical texts: predicting and defining unfamiliar terms. AMIA Annu Symp Proc. 2007; :239-43. PMC: 1839621. View

15.

Trafimow D, Sheeran P, Conner M, Finlay K . Evidence that perceived behavioural control is a multidimensional construct: perceived control and perceived difficulty. Br J Soc Psychol. 2002; 41(Pt 1):101-21. DOI: 10.1348/014466602165081. View

16.

DAlessandro D, Kingsley P . The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med. 2001; 155(7):807-12. DOI: 10.1001/archpedi.155.7.807. View

17.

Berland G, Elliott M, Morales L, Algazy J, Kravitz R, Broder M . Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001; 285(20):2612-21. PMC: 4182102. DOI: 10.1001/jama.285.20.2612. View

18.

Root J, Stableford S . Easy-to-read consumer communications: a missing link in Medicaid managed care. J Health Polit Policy Law. 1999; 24(1):1-26. DOI: 10.1215/03616878-24-1-1. View

19.

Baker D, Gazmararian J, Williams M, Scott T, Parker R, Green D . Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health. 2002; 92(8):1278-83. PMC: 1447230. DOI: 10.2105/ajph.92.8.1278. View