Novel Approach to Personalized Physician Recommendations Using Semantic Features and Response Metrics: Model Evaluation Study

Overview

Journal JMIR Hum Factors

Specialty Health Services

Date 2024 Aug 15

PMID 39146009

Authors

Yingbin Zheng

Yunping Cai

Yiwei Yan

Sai Chen

Kai Gong

Affiliations

Soon will be listed here.

Abstract

Background: The rapid growth of web-based medical services has highlighted the significance of smart triage systems in helping patients find the most appropriate physicians. However, traditional triage methods often rely on department recommendations and are insufficient to accurately match patients' textual questions with physicians' specialties. Therefore, there is an urgent need to develop algorithms for recommending physicians.

Objective: This study aims to develop and validate a patient-physician hybrid recommendation (PPHR) model with response metrics for better triage performance.

Methods: A total of 646,383 web-based medical consultation records from the Internet Hospital of the First Affiliated Hospital of Xiamen University were collected. Semantic features representing patients and physicians were developed to identify the set of most similar questions and semantically expand the pool of recommended physician candidates, respectively. The physicians' response rate feature was designed to improve candidate rankings. These 3 characteristics combine to create the PPHR model. Overall, 5 physicians participated in the evaluation of the efficiency of the PPHR model through multiple metrics and questionnaires as well as the performance of Sentence Bidirectional Encoder Representations from Transformers and Doc2Vec in text embedding.

Results: The PPHR model reaches the best recommendation performance when the number of recommended physicians is 14. At this point, the model has an F-score of 76.25%, a proportion of high-quality services of 41.05%, and a rating of 3.90. After removing physicians' characteristics and response rates from the PPHR model, the F-score decreased by 12.05%, the proportion of high-quality services fell by 10.87%, the average hit ratio dropped by 1.06%, and the rating declined by 11.43%. According to whether those 5 physicians were recommended by the PPHR model, Sentence Bidirectional Encoder Representations from Transformers achieved an average hit ratio of 88.6%, while Doc2Vec achieved an average hit ratio of 53.4%.

Conclusions: The PPHR model uses semantic features and response metrics to enable patients to accurately find the physician who best suits their needs.

References

Wang Y, Sun Y, Wang B, Wu Z, He X, Zhao Y . Transfer learning for clustering single-cell RNA-seq data crossing-species and batch, case on uterine fibroids. Brief Bioinform. 2023; 25(1). PMC: 10664408. DOI: 10.1093/bib/bbad426. View

Li F, Yu H . ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network. Proc AAAI Conf Artif Intell. 2021; 34(5):8180-8187. PMC: 8315310. DOI: 10.1609/aaai.v34i05.6331. View

Ju C, Zhang S . Doctor Recommendation Model Based on Ontology Characteristics and Disease Text Mining Perspective. Biomed Res Int. 2021; 2021:7431199. PMC: 8379386. DOI: 10.1155/2021/7431199. View

Neugebauer J, Tothova V, Dolezalova J . Use of Standardized and Non-Standardized Tools for Measuring the Risk of Falls and Independence in Clinical Practice. Int J Environ Res Public Health. 2021; 18(6). PMC: 8004039. DOI: 10.3390/ijerph18063226. View

Fay M, Malinovsky Y . Confidence intervals of the Mann-Whitney parameter that are compatible with the Wilcoxon-Mann-Whitney test. Stat Med. 2018; 37(27):3991-4006. PMC: 6377068. DOI: 10.1002/sim.7890. View

Chen W, Zhong C, Peng J, Wei Z . DxFormer: a decoupled automatic diagnostic system based on decoder-encoder transformer with dense symptom representations. Bioinformatics. 2022; 39(1). PMC: 9825744. DOI: 10.1093/bioinformatics/btac744. View

Argote L . Knowledge Transfer Within Organizations: Mechanisms, Motivation, and Consideration. Annu Rev Psychol. 2023; 75:405-431. DOI: 10.1146/annurev-psych-022123-105424. View

Li C, Liu J, Qian G, Wang Z, Han J . Double chain system for online and offline medical data sharing private and consortium blockchain: A system design study. Front Public Health. 2022; 10:1012202. PMC: 9595571. DOI: 10.3389/fpubh.2022.1012202. View

Zhang L, Wang X, Xiao H, Ma C, Li X, Dai G . Governance mechanisms for chronic disease diagnosis and treatment systems in the post-pandemic era. Front Public Health. 2022; 10:1023022. PMC: 9792788. DOI: 10.3389/fpubh.2022.1023022. View

10.

Lu W, Zhai Y . Self-Adaptive Telemedicine Specialist Recommendation Considering Specialist Activity and Patient Feedback. Int J Environ Res Public Health. 2022; 19(9). PMC: 9101090. DOI: 10.3390/ijerph19095594. View

11.

Sexton V, Dale J, Bryce C, Barry J, Sellers E, Atherton H . Service use, clinical outcomes and user experience associated with urgent care services that use telephone-based digital triage: a systematic review. BMJ Open. 2022; 12(1):e051569. PMC: 8724705. DOI: 10.1136/bmjopen-2021-051569. View

12.

Haleem A, Javaid M, Singh R, Suman R . Telemedicine for healthcare: Capabilities, features, barriers, and applications. Sens Int. 2021; 2:100117. PMC: 8590973. DOI: 10.1016/j.sintl.2021.100117. View

13.

Xu S, Leng Y, Feng G, Zhang C, Chen M . A gene pathway enrichment method based on improved TF-IDF algorithm. Biochem Biophys Rep. 2023; 34:101421. PMC: 10009669. DOI: 10.1016/j.bbrep.2023.101421. View

14.

Liu J, Li C, Huang Y, Han J . An intelligent medical guidance and recommendation model driven by patient-physician communication data. Front Public Health. 2023; 11:1098206. PMC: 9909411. DOI: 10.3389/fpubh.2023.1098206. View

15.

Mosadeghrad A . Factors Affecting Medical Service Quality. Iran J Public Health. 2015; 43(2):210-20. PMC: 4450689. View

16.

Fekonja U, Strnad M, Fekonja Z . Association between triage nurses' job satisfaction and professional capability: Results of a mixed-method study. J Nurs Manag. 2022; 30(8):4364-4377. PMC: 10091795. DOI: 10.1111/jonm.13860. View

17.

Bayeh Y, Tsehay C, Negash W . Health system responsiveness and associated factors for delivery care in public health facilities, Dessie City Administration, South Wollo zone, Ethiopia: Cross-sectional study design. BMJ Open. 2023; 13(7):e069655. PMC: 10364148. DOI: 10.1136/bmjopen-2022-069655. View

18.

Liu X, Esser D, Wagstaff B, Zavodni A, Matsuura N, Kelly J . Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning. Sci Rep. 2022; 12(1):21130. PMC: 9729303. DOI: 10.1038/s41598-022-25572-w. View

19.

Liu T, Cheng C, Hsu C, Chaou C, Ng C, Jeng M . Validation of a five-level triage system in pediatric trauma and the effectiveness of triage nurse modification: A multi-center cohort analysis. Front Med (Lausanne). 2022; 9:947501. PMC: 9664936. DOI: 10.3389/fmed.2022.947501. View

20.

Kuperwajs I, Schutt H, Ma W . Using deep neural networks as a guide for modeling human planning. Sci Rep. 2023; 13(1):20269. PMC: 10662256. DOI: 10.1038/s41598-023-46850-1. View