Assessing the Effectiveness of ChatGPT in Delivering Mental Health Support: A Qualitative Study

Overview

Journal J Multidiscip Healthc

Publisher Dove Medical Press

Specialty Health Services

Date 2024 Feb 5

PMID 38314011

Authors

Fahad Alanezi

Affiliations

Soon will be listed here.

Abstract

Background: Artificial Intelligence (AI) applications are widely researched for their potential in effectively improving the healthcare operations and disease management. However, the research trend shows that these applications also have significant negative implications on the service delivery.

Purpose: To assess the use of ChatGPT for mental health support.

Methods: Due to the novelty and unfamiliarity of the ChatGPT technology, a quasi-experimental design was chosen for this study. Outpatients from a public hospital were included in the sample. A two-week experiment followed by semi-structured interviews was conducted in which participants used ChatGPT for mental health support. Semi-structured interviews were conducted with 24 individuals with mental health conditions.

Results: Eight positive factors (psychoeducation, emotional support, goal setting and motivation, referral and resource information, self-assessment and monitoring, cognitive behavioral therapy, crisis interventions, and psychotherapeutic exercises) and four negative factors (ethical and legal considerations, accuracy and reliability, limited assessment capabilities, and cultural and linguistic considerations) were associated with the use of ChatGPT for mental health support.

Conclusion: It is important to carefully consider the ethical, reliability, accuracy, and legal challenges and develop appropriate strategies to mitigate them in order to ensure safe and effective use of AI-based applications like ChatGPT in mental health support.

Citing Articles

Exploring the Ethical Challenges of Conversational AI in Mental Health Care: Scoping Review.

Rahsepar Meadi M, Sillekens T, Metselaar S, van Balkom A, Bernstein J, Batelaan N JMIR Ment Health. 2025; 12:e60432.

PMID: 39983102 PMC: 11890142. DOI: 10.2196/60432.

Healthcare professionals and the public sentiment analysis of ChatGPT in clinical practice.

Lu L, Zhu Y, Yang J, Yang Y, Ye J, Ai S Sci Rep. 2025; 15(1):1223.

PMID: 39774168 PMC: 11707298. DOI: 10.1038/s41598-024-84512-y.

Effects of Large Language Model-Based Offerings on the Well-Being of Students: Qualitative Study.

Selim R, Basu A, Anto A, Foscht T, Eisingerich A JMIR Form Res. 2024; 8():e64081.

PMID: 39729617 PMC: 11724218. DOI: 10.2196/64081.

The ethical aspects of integrating sentiment and emotion analysis in chatbots for depression intervention.

Denecke K, Gabarron E Front Psychiatry. 2024; 15:1462083.

PMID: 39611131 PMC: 11602467. DOI: 10.3389/fpsyt.2024.1462083.

Integrating large language models in mental health practice: a qualitative descriptive study based on expert interviews.

Ma Y, Zeng Y, Liu T, Sun R, Xiao M, Wang J Front Public Health. 2024; 12:1475867.

PMID: 39559378 PMC: 11571062. DOI: 10.3389/fpubh.2024.1475867.

References

Nov O, Singh N, Mann D . Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study. JMIR Med Educ. 2023; 9:e46939. PMC: 10366957. DOI: 10.2196/46939. View

Goodman R, Patrinely Jr J, Osterman T, Wheless L, Johnson D . On the cusp: Considering the impact of artificial intelligence language models in healthcare. Med. 2023; 4(3):139-140. DOI: 10.1016/j.medj.2023.02.008. View

Uludag K, Wang D, Goodman C, Chen D, Wang L, Zhang X . Prevalence, clinical correlates and risk factors associated with Tardive Dyskinesia in Chinese patients with schizophrenia. Asian J Psychiatr. 2021; 66:102877. DOI: 10.1016/j.ajp.2021.102877. View

Dosovitsky G, Pineda B, Jacobson N, Chang C, Escoredo M, Bunge E . Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage. JMIR Form Res. 2020; 4(11):e17065. PMC: 7695525. DOI: 10.2196/17065. View

Naughton M, Weaver K . Physical and mental health among cancer survivors: considerations for long-term care and quality of life. N C Med J. 2014; 75(4):283-6. PMC: 4503227. DOI: 10.18043/ncm.75.4.283. View

Farhat F . ChatGPT as a Complementary Mental Health Resource: A Boon or a Bane. Ann Biomed Eng. 2023; 52(5):1111-1114. DOI: 10.1007/s10439-023-03326-7. View

Heinz A, Zhao X, Liu S . Implications of the Association of Social Exclusion With Mental Health. JAMA Psychiatry. 2019; 77(2):113-114. DOI: 10.1001/jamapsychiatry.2019.3009. View

Clapp M, Aurora N, Herrera L, Bhatia M, Wilen E, Wakefield S . Gut microbiota's effect on mental health: The gut-brain axis. Clin Pract. 2017; 7(4):987. PMC: 5641835. DOI: 10.4081/cp.2017.987. View

Dworkin S . Sample size policy for qualitative studies using in-depth interviews. Arch Sex Behav. 2012; 41(6):1319-20. DOI: 10.1007/s10508-012-0016-6. View

10.

Baker D, Wolf M, Feinglass J, Thompson J, Gazmararian J, Huang J . Health literacy and mortality among elderly persons. Arch Intern Med. 2007; 167(14):1503-9. DOI: 10.1001/archinte.167.14.1503. View

11.

Han J, Park J, Lee H . Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. BMC Med Educ. 2022; 22(1):830. PMC: 9713176. DOI: 10.1186/s12909-022-03898-3. View

12.

Chesney E, Goodwin G, Fazel S . Risks of all-cause and suicide mortality in mental disorders: a meta-review. World Psychiatry. 2014; 13(2):153-60. PMC: 4102288. DOI: 10.1002/wps.20128. View

13.

Spallek S, Birrell L, Kershaw S, Devine E, Thornton L . Can we use ChatGPT for Mental Health and Substance Use Education? Examining Its Quality and Potential Harms. JMIR Med Educ. 2023; 9:e51243. PMC: 10722374. DOI: 10.2196/51243. View

14.

Shahsavar Y, Choudhury A . User Intentions to Use ChatGPT for Self-Diagnosis and Health-Related Purposes: Cross-sectional Survey Study. JMIR Hum Factors. 2023; 10:e47564. PMC: 10233444. DOI: 10.2196/47564. View

15.

Berkman N, Sheridan S, Donahue K, Halpern D, Crotty K . Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011; 155(2):97-107. DOI: 10.7326/0003-4819-155-2-201107190-00005. View

16.

Cascella M, Montomoli J, Bellini V, Bignami E . Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. J Med Syst. 2023; 47(1):33. PMC: 9985086. DOI: 10.1007/s10916-023-01925-4. View

17.

DeWalt D, Dilling M, Rosenthal M, Pignone M . Low parental literacy is associated with worse asthma care measures in children. Ambul Pediatr. 2007; 7(1):25-31. PMC: 1797805. DOI: 10.1016/j.ambp.2006.10.001. View

18.

Romeo R . The impact of stress on the structure of the adolescent brain: Implications for adolescent mental health. Brain Res. 2016; 1654(Pt B):185-191. DOI: 10.1016/j.brainres.2016.03.021. View

19.

Biswas S . Role of Chat GPT in Public Health. Ann Biomed Eng. 2023; 51(5):868-869. DOI: 10.1007/s10439-023-03172-7. View

20.

Orpana H, Vachon J, Dykxhoorn J, McRae L, Jayaraman G . Monitoring positive mental health and its determinants in Canada: the development of the Positive Mental Health Surveillance Indicator Framework. Health Promot Chronic Dis Prev Can. 2016; 36(1):1-10. PMC: 4939463. DOI: 10.24095/hpcdp.36.1.01. View