The Current State and Validity of Digital Assessment Tools for Psychiatry: Systematic Review

Overview

Journal JMIR Ment Health

Specialties Health Services
Psychiatry
Psychology

Date 2022 Mar 30

PMID 35353053

Authors

Nayra A Martin-Key

Benedetta Spadaro

Erin Funnell

Eleanor Jane Barker

Thea Sofie Schei

Jakub Tomasik

Sabine Bahn

Affiliations

Soon will be listed here.

Abstract

Background: Given the role digital technologies are likely to play in the future of mental health care, there is a need for a comprehensive appraisal of the current state and validity (ie, screening or diagnostic accuracy) of digital mental health assessments.

Objective: The aim of this review is to explore the current state and validity of question-and-answer-based digital tools for diagnosing and screening psychiatric conditions in adults.

Methods: This systematic review was based on the Population, Intervention, Comparison, and Outcome framework and was carried out in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. MEDLINE, Embase, Cochrane Library, ASSIA, Web of Science Core Collection, CINAHL, and PsycINFO were systematically searched for articles published between 2005 and 2021. A descriptive evaluation of the study characteristics and digital solutions and a quantitative appraisal of the screening or diagnostic accuracy of the included tools were conducted. Risk of bias and applicability were assessed using the revised tool for the Quality Assessment of Diagnostic Accuracy Studies 2.

Results: A total of 28 studies met the inclusion criteria, with the most frequently evaluated conditions encompassing generalized anxiety disorder, major depressive disorder, and any depressive disorder. Most of the studies used digitized versions of existing pen-and-paper questionnaires, with findings revealing poor to excellent screening or diagnostic accuracy (sensitivity=0.32-1.00, specificity=0.37-1.00, area under the receiver operating characteristic curve=0.57-0.98) and a high risk of bias for most of the included studies.

Conclusions: The field of digital mental health tools is in its early stages, and high-quality evidence is lacking.

International Registered Report Identifier (irrid): RR2-10.2196/25382.

Citing Articles

Prevalence and associated factors of intimate partner violence against pregnant women who attend antenatal care in Denmark and Spain: A digital screening approach.

Fernandez-Lopez R, Andreasen K, Ankerstjerne L, Martin-de-Las-Heras S, Rasch V, Megias J Acta Obstet Gynecol Scand. 2024; 104(1):139-150.

PMID: 39509551 PMC: 11683538. DOI: 10.1111/aogs.15000.

INNOVATIONS IN DIGITAL HEALTH TO IMPROVE CARE DELIVERY: THE BJC HEALTHCARE/WASHINGTON UNIVERSITY SCHOOL OF MEDICINE HEALTHCARE INNOVATION LAB.

Maddox T Trans Am Clin Climatol Assoc. 2024; 134:239-251.

PMID: 39135571 PMC: 11316891.

Uncertainty Breeds Anxiety and Depression: The Impact of the Russian Invasion in Ukraine on a Swedish Clinical Population Receiving Internet-Based Psychotherapy.

Hlynsson J, Gustafsson O, Carlbring P Clin Psychol Eur. 2024; 6(1):e12083.

PMID: 39119223 PMC: 11303911. DOI: 10.32872/cpe.12083.

Diagnostic accuracy and clinical utility of the PHQ-2 and GAD-2: a comparison with long-format measures for depression and anxiety.

Hlynsson J, Carlbring P Front Psychol. 2024; 15:1259997.

PMID: 38800678 PMC: 11123884. DOI: 10.3389/fpsyg.2024.1259997.

Developing and Implementing a Web-Based Branching Logic Survey to Support Psychiatric Crisis Evaluations of Individuals With Developmental Disabilities: Qualitative Study and Evaluation of Validity.

Bilder D, Mthembu M, Worsham W, Aguayo P, Knight J, Deng S JMIR Ment Health. 2024; 11:e50907.

PMID: 38551644 PMC: 11015367. DOI: 10.2196/50907.

References

Gottschalk M, Cooper J, Chan M, Bot M, Penninx B, Bahn S . Discovery of serum biomarkers predicting development of a subsequent depressive episode in social anxiety disorder. Brain Behav Immun. 2015; 48:123-31. DOI: 10.1016/j.bbi.2015.04.011. View

Moher D, Liberati A, Tetzlaff J, Altman D . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7):e1000097. PMC: 2707599. DOI: 10.1371/journal.pmed.1000097. View

Oromendia P, Bonillo A, Molinuevo B . Web-based screening for Panic Disorder: Validity of a single-item instrument. J Affect Disord. 2015; 180:138-41. DOI: 10.1016/j.jad.2015.03.061. View

Tomasik J, Han S, Barton-Owen G, Mirea D, Martin-Key N, Rustogi N . A machine learning algorithm to differentiate bipolar disorder from major depressive disorder using an online mental health questionnaire and blood biomarker data. Transl Psychiatry. 2021; 11(1):41. PMC: 7804187. DOI: 10.1038/s41398-020-01181-x. View

Du N, Yu K, Ye Y, Chen S . Validity study of Patient Health Questionnaire-9 items for Internet screening in depression among Chinese university students. Asia Pac Psychiatry. 2017; 9(3). PMC: 5580087. DOI: 10.1111/appy.12266. View

van Ballegooijen W, Riper H, Cuijpers P, van Oppen P, Smit J . Validation of online psychometric instruments for common mental health disorders: a systematic review. BMC Psychiatry. 2016; 16:45. PMC: 4766750. DOI: 10.1186/s12888-016-0735-7. View

Gibbons R, Weiss D, Pilkonis P, Frank E, Moore T, Kim J . Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiatry. 2013; 171(2):187-94. PMC: 4052830. DOI: 10.1176/appi.ajp.2013.13020178. View

Ter Huurne E, de Haan H, Ten Napel-Schutz M, Postel M, Menting J, van der Palen J . Is the Eating Disorder Questionnaire-Online (EDQ-O) a valid diagnostic instrument for the DSM-IV-TR classification of eating disorders?. Compr Psychiatry. 2014; 57:167-76. DOI: 10.1016/j.comppsych.2014.10.019. View

Schulte-van Maaren Y, Giltay E, van Hemert A, Zitman F, de Waal M, Carlier I . Reference values for anxiety questionnaires: the Leiden Routine Outcome Monitoring Study. J Affect Disord. 2013; 150(3):1008-18. DOI: 10.1016/j.jad.2013.05.031. View

10.

Nguyen D, Klein B, Meyer D, Austin D, Abbott J . The Diagnostic Validity and Reliability of an Internet-Based Clinical Assessment Program for Mental Disorders. J Med Internet Res. 2015; 17(9):e218. PMC: 4642400. DOI: 10.2196/jmir.4195. View

11.

Kroenke K, Spitzer R, Williams J . The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001; 16(9):606-13. PMC: 1495268. DOI: 10.1046/j.1525-1497.2001.016009606.x. View

12.

Germund Nielsen M, Ornbol E, Bech P, Vestergaard M, Sparle Christensen K . The criterion validity of the web-based Major Depression Inventory when used on clinical suspicion of depression in primary care. Clin Epidemiol. 2017; 9:355-365. PMC: 5505538. DOI: 10.2147/CLEP.S132913. View

13.

Han S, Tomasik J, Rustogi N, Lago S, Barton-Owen G, Eljasz P . Diagnostic prediction model development using data from dried blood spot proteomics and a digital mental health assessment to identify major depressive disorder among individuals presenting with low mood. Brain Behav Immun. 2020; 90:184-195. DOI: 10.1016/j.bbi.2020.08.011. View

14.

Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E . Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014; 13(3):288-95. PMC: 4219070. DOI: 10.1002/wps.20151. View

15.

McNeely J, Cleland C, Strauss S, Palamar J, Rotrosen J, Saitz R . Validation of Self-Administered Single-Item Screening Questions (SISQs) for Unhealthy Alcohol and Drug Use in Primary Care Patients. J Gen Intern Med. 2015; 30(12):1757-64. PMC: 4636560. DOI: 10.1007/s11606-015-3391-6. View

16.

Meuldijk D, Giltay E, Carlier I, van Vliet I, van Hemert A, Zitman F . A Validation Study of the Web Screening Questionnaire (WSQ) Compared With the Mini-International Neuropsychiatric Interview-Plus (MINI-Plus). JMIR Ment Health. 2017; 4(3):e35. PMC: 5596296. DOI: 10.2196/mental.5453. View

17.

Donker T, van Straten A, Marks I, Cuijpers P . Brief self-rated screening for depression on the Internet. J Affect Disord. 2009; 122(3):253-9. DOI: 10.1016/j.jad.2009.07.013. View

18.

Fowler J, Madan A, Allen J, Patriquin M, Sharp C, Oldham J . Clinical utility of the DSM-5 alternative model for borderline personality disorder: Differential diagnostic accuracy of the BFI, SCID-II-PQ, and PID-5. Compr Psychiatry. 2017; 80:97-103. DOI: 10.1016/j.comppsych.2017.09.003. View

19.

Hanley J, McNeil B . A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983; 148(3):839-43. DOI: 10.1148/radiology.148.3.6878708. View

20.

Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V . Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012; 380(9859):2095-128. PMC: 10790329. DOI: 10.1016/S0140-6736(12)61728-0. View