Automated Extraction of Post-stroke Functional Outcomes from Unstructured Electronic Health Records

Overview

Journal Eur Stroke J

Date 2025 Jan 22

PMID 39838914

Authors

Marta Fernandes

Kaileigh Gallagher

Niels Turley

Aditya Gupta

M Brandon Westover

Aneesh B Singhal

Sahar F Zafar

Affiliations

Soon will be listed here.

Abstract

Purpose: Population level tracking of post-stroke functional outcomes is critical to guide interventions that reduce the burden of stroke-related disability. However, functional outcomes are often missing or documented in unstructured notes. We developed a natural language processing (NLP) model that reads electronic health records (EHR) notes to automatically determine the modified Rankin Scale (mRS).

Method: We included consecutive patients (⩾18 years) with acute stroke admitted to our center (2015-2024). mRS scores were obtained from the Get With the Guidelines registry and clinical notes (if documented), and used as the gold standard to compare against NLP-generated scores. We used text-based features from notes, along with age, sex, discharge status, and outpatient follow-up to train a logistic regression for prediction of good (0-2) versus poor (3-6) mRS, and a linear regression for the full range of mRS scores. The models were trained for prediction of mRS at hospital discharge and post-discharge. The models were externally validated in a dataset of patients with brain injuries from a different healthcare center.

Findings: We included 5307 patients, 5006 in train and test and 301 in validation; average age was 69 (SD 15) and 65 (SD 17) years, respectively; 47% female. The logistic regression achieved an area under the receiver operating curve (AUROC) of 0.94 [CI 0.93-0.95] (test) and 0.94 [0.91-0.96] (validation), and the linear model a root mean squared error (RMSE) of 0.91 [0.87-0.94] (test) and 1.17 [1.06-1.28] (validation).

Discussion And Conclusion: The NLP-based model is suitable for use in large-scale phenotyping of stroke functional outcomes and population health research.

References

Brugnara G, Neuberger U, Mahmutoglu M, Foltyn M, Herweh C, Nagel S . Multimodal Predictive Modeling of Endovascular Treatment Outcome for Acute Ischemic Stroke Using Machine-Learning. Stroke. 2020; 51(12):3541-3551. DOI: 10.1161/STROKEAHA.120.030287. View

van Os H, Ramos L, Hilbert A, van Leeuwen M, van Walderveen M, Kruyt N . Predicting Outcome of Endovascular Treatment for Acute Ischemic Stroke: Potential Value of Machine Learning Algorithms. Front Neurol. 2018; 9:784. PMC: 6167479. DOI: 10.3389/fneur.2018.00784. View

Asadi H, Dowling R, Yan B, Mitchell P . Machine learning for outcome prediction of acute ischemic stroke post intra-arterial therapy. PLoS One. 2014; 9(2):e88225. PMC: 3919736. DOI: 10.1371/journal.pone.0088225. View

Heo J, Yoon J, Park H, Kim Y, Nam H, Heo J . Machine Learning-Based Model for Prediction of Outcomes in Acute Stroke. Stroke. 2019; 50(5):1263-1265. DOI: 10.1161/STROKEAHA.118.024293. View

Collins G, Moons K, Dhiman P, Riley R, Beam A, Van Calster B . TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024; 385:e078378. PMC: 11019967. DOI: 10.1136/bmj-2023-078378. View

Renedo D, Acosta J, Leasure A, Sharma R, Krumholz H, de Havenon A . Burden of Ischemic and Hemorrhagic Stroke Across the US From 1990 to 2019. JAMA Neurol. 2024; . PMC: 10913004. DOI: 10.1001/jamaneurol.2024.0190. View

Feigin V, Brainin M, Norrving B, Martins S, Sacco R, Hacke W . World Stroke Organization (WSO): Global Stroke Fact Sheet 2022. Int J Stroke. 2022; 17(1):18-29. DOI: 10.1177/17474930211065917. View

Reeves M, Smith E, Fonarow G, Zhao X, Thompson M, Peterson E . Variation and Trends in the Documentation of National Institutes of Health Stroke Scale in GWTG-Stroke Hospitals. Circ Cardiovasc Qual Outcomes. 2015; 8(6 Suppl 3):S90-8. DOI: 10.1161/CIRCOUTCOMES.115.001775. View

Rajsic S, Gothe H, Borba H, Sroczynski G, Vujicic J, Toell T . Economic burden of stroke: a systematic review on post-stroke care. Eur J Health Econ. 2018; 20(1):107-134. DOI: 10.1007/s10198-018-0984-0. View

10.

Coleman C, Concha M, Koch B, Lovelace B, Christoph M, Cohen A . Derivation and validation of a composite scoring system (SAVED) for prediction of unfavorable modified Rankin scale score following intracerebral hemorrhage. Front Neurol. 2023; 14:1112723. PMC: 9992975. DOI: 10.3389/fneur.2023.1112723. View

11.

Quinn T, Ray G, Atula S, Walters M, Dawson J, Lees K . Deriving modified Rankin scores from medical case-records. Stroke. 2008; 39(12):3421-3. DOI: 10.1161/STROKEAHA.108.519306. View

12.

Ramos L, Kappelhof M, van Os H, Chalos V, van Kranendonk K, Kruyt N . Predicting Poor Outcome Before Endovascular Treatment in Patients With Acute Ischemic Stroke. Front Neurol. 2020; 11:580957. PMC: 7593486. DOI: 10.3389/fneur.2020.580957. View

13.

Fernandes M, Valizadeh N, Alabsi H, Quadri S, Tesh R, Bucklin A . Classification of neurologic outcomes from medical notes using natural language processing. Expert Syst Appl. 2023; 214. PMC: 9974159. DOI: 10.1016/j.eswa.2022.119171. View

14.

Xie Y, Jiang B, Gong E, Li Y, Zhu G, Michel P . JOURNAL CLUB: Use of Gradient Boosting Machine Learning to Predict Patient Outcome in Acute Ischemic Stroke on the Basis of Imaging, Demographic, and Clinical Information. AJR Am J Roentgenol. 2018; 212(1):44-51. DOI: 10.2214/AJR.18.20260. View

15.

Alaka S, Menon B, Brobbey A, Williamson T, Goyal M, Demchuk A . Functional Outcome Prediction in Ischemic Stroke: A Comparison of Machine Learning Algorithms and Regression Models. Front Neurol. 2020; 11:889. PMC: 7479334. DOI: 10.3389/fneur.2020.00889. View

16.

Hung L, Su Y, Sun J, Huang W, Sung S . Clinical narratives as a predictor for prognosticating functional outcomes after intracerebral hemorrhage. J Neurol Sci. 2023; 453:120807. DOI: 10.1016/j.jns.2023.120807. View

17.

Zhang M, Mlynash M, Sainani K, Albers G, Lansberg M . Ordinal Prediction Model of 90-Day Modified Rankin Scale in Ischemic Stroke. Front Neurol. 2021; 12:727171. PMC: 8569127. DOI: 10.3389/fneur.2021.727171. View

18.

Sung S, Chen C, Pan R, Hu Y, Jeng J . Natural Language Processing Enhances Prediction of Functional Outcome After Acute Ischemic Stroke. J Am Heart Assoc. 2021; 10(24):e023486. PMC: 9075227. DOI: 10.1161/JAHA.121.023486. View

19.

Herzog L, Kook L, Hamann J, Globas C, Heldner M, Seiffge D . Deep Learning Versus Neurologists: Functional Outcome Prediction in LVO Stroke Patients Undergoing Mechanical Thrombectomy. Stroke. 2023; 54(7):1761-1769. DOI: 10.1161/STROKEAHA.123.042496. View

20.

Kruse C, Stein A, Thomas H, Kaur H . The use of Electronic Health Records to Support Population Health: A Systematic Review of the Literature. J Med Syst. 2018; 42(11):214. PMC: 6182727. DOI: 10.1007/s10916-018-1075-6. View