Emotion Recognition Support System: Where Physicians and Psychiatrists Meet Linguists and Data Engineers

Overview

Journal World J Psychiatry

Publisher Baishideng Publishing Group

Specialty Psychiatry

Date 2023 Jan 23

PMID 36687372

Authors

Peyman Adibi

Simindokht Kalani

Sayed Jalal Zahabi

Homa Asadi

Mohsen Bakhtiar

Mohammad Reza Heidarpour

Hamidreza Roohafza

Hassan Shahoon

Mohammad Amouzadeh

Affiliations

Soon will be listed here.

Abstract

An important factor in the course of daily medical diagnosis and treatment is understanding patients' emotional states by the caregiver physicians. However, patients usually avoid speaking out their emotions when expressing their somatic symptoms and complaints to their non-psychiatrist doctor. On the other hand, clinicians usually lack the required expertise (or time) and have a deficit in mining various verbal and non-verbal emotional signals of the patients. As a result, in many cases, there is an emotion recognition barrier between the clinician and the patients making all patients seem the same except for their different somatic symptoms. In particular, we aim to identify and combine three major disciplines (psychology, linguistics, and data science) approaches for detecting emotions from verbal communication and propose an integrated solution for emotion recognition support. Such a platform may give emotional guides and indices to the clinician based on verbal communication at the consultation time.

References

Wang L, Bastiaansen M, Yang Y, Hagoort P . ERP evidence on the interaction between information structure and emotional salience of words. Cogn Affect Behav Neurosci. 2013; 13(2):297-310. DOI: 10.3758/s13415-012-0146-2. View

Rosenberg S, Tucker G . Verbal behavior and schizophrenia. The semantic dimension. Arch Gen Psychiatry. 1979; 36(12):1331-7. DOI: 10.1001/archpsyc.1979.01780120061008. View

Kim S, Kaplowitz S, Johnston M . The effects of physician empathy on patient satisfaction and compliance. Eval Health Prof. 2004; 27(3):237-51. DOI: 10.1177/0163278704267037. View

DiMatteo M, Hays R, Prince L . Relationship of physicians' nonverbal communication skill to patient satisfaction, appointment noncompliance, and physician workload. Health Psychol. 1986; 5(6):581-94. DOI: 10.1037//0278-6133.5.6.581. View

Mundt J, Snyder P, Cannizzaro M, Chappie K, Geralts D . Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. J Neurolinguistics. 2011; 20(1):50-64. PMC: 3022333. DOI: 10.1016/j.jneuroling.2006.04.001. View

Blue A, Chessman A, Gilbert G, Mainous 3rd A . Responding to patients' emotions: important for standardized patient satisfaction. Fam Med. 2000; 32(5):326-30. View

Sobin C, Alpert M . Emotion in speech: the acoustic attributes of fear, anger, sadness, and joy. J Psycholinguist Res. 1999; 28(4):347-65. DOI: 10.1023/a:1023237014909. View

Stevanovic M, Perakyla A . Experience sharing, emotional reciprocity, and turn-taking. Front Psychol. 2015; 6:450. PMC: 4396131. DOI: 10.3389/fpsyg.2015.00450. View

Del Piccolo L, Goss C, Zimmermann C . The Third Meeting of the Verona Network on Sequence Analysis. Finding common grounds in defining patient cues and concerns and the appropriateness of provider responses. Patient Educ Couns. 2005; 57(2):241-4. DOI: 10.1016/j.pec.2005.03.003. View

10.

Riedl D, Schussler G . The Influence of Doctor-Patient Communication on Health Outcomes: A Systematic Review. Z Psychosom Med Psychother. 2017; 63(2):131-150. DOI: 10.13109/zptm.2017.63.2.131. View

11.

Banse R, Scherer K . Acoustic profiles in vocal emotion expression. J Pers Soc Psychol. 1996; 70(3):614-36. DOI: 10.1037//0022-3514.70.3.614. View

12.

Satterfield J, Hughes E . Emotion skills training for medical students: a systematic review. Med Educ. 2007; 41(10):935-41. DOI: 10.1111/j.1365-2923.2007.02835.x. View

13.

M Thompson B, Teal C, Scott S, Manning S, Greenfield E, Shada R . Following the clues: teaching medical students to explore patients' contexts. Patient Educ Couns. 2010; 80(3):345-50. PMC: 3049898. DOI: 10.1016/j.pec.2010.06.035. View

14.

Finset A . "I am worried, Doctor!" Emotions in the doctor-patient relationship. Patient Educ Couns. 2012; 88(3):359-63. DOI: 10.1016/j.pec.2012.06.022. View

15.

DiMatteo M, Taranta A, Friedman H, Prince L . Predicting patient satisfaction from physicians' nonverbal communication skills. Med Care. 1980; 18(4):376-87. DOI: 10.1097/00005650-198004000-00003. View

16.

Murray I, Arnott J . Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. J Acoust Soc Am. 1993; 93(2):1097-108. DOI: 10.1121/1.405558. View

17.

Ogle J, Bushnell J, Caputi P . Empathy is related to clinical competence in medical care. Med Educ. 2013; 47(8):824-31. DOI: 10.1111/medu.12232. View

18.

Bitouk D, Verma R, Nenkova A . Class-Level Spectral Features for Emotion Recognition. Speech Commun. 2013; 52(7-8):613-625. PMC: 3686526. DOI: 10.1016/j.specom.2010.02.010. View

19.

Alpert M, Pouget E, Silva R . Reflections of depression in acoustic measures of the patient's speech. J Affect Disord. 2001; 66(1):59-69. DOI: 10.1016/s0165-0327(00)00335-9. View

20.

Levinson W, Gorawara-Bhat R, Lamb J . A study of patient clues and physician responses in primary care and surgical settings. JAMA. 2000; 284(8):1021-7. DOI: 10.1001/jama.284.8.1021. View