AskHERMES: An Online Question Answering System for Complex Clinical Questions

Overview

Journal J Biomed Inform

Publisher Elsevier

Specialty Medical Informatics

Date 2011 Jan 25

PMID 21256977

Citations 45

Authors

Yonggang Cao

Feifan Liu

Pippa Simpson

Lamont Antieau

Andrew Bennett

James J Cimino

John Ely

Hong Yu

Affiliations

Soon will be listed here.

Abstract

Objective: Clinical questions are often long and complex and take many forms. We have built a clinical question answering system named AskHERMES to perform robust semantic analysis on complex clinical questions and output question-focused extractive summaries as answers.

Design: This paper describes the system architecture and a preliminary evaluation of AskHERMES, which implements innovative approaches in question analysis, summarization, and answer presentation. Five types of resources were indexed in this system: MEDLINE abstracts, PubMed Central full-text articles, eMedicine documents, clinical guidelines and Wikipedia articles.

Measurement: We compared the AskHERMES system with Google (Google and Google Scholar) and UpToDate and asked physicians to score the three systems by ease of use, quality of answer, time spent, and overall performance.

Results: AskHERMES allows physicians to enter a question in a natural way with minimal query formulation and allows physicians to efficiently navigate among all the answer sentences to quickly meet their information needs. In contrast, physicians need to formulate queries to search for information in Google and UpToDate. The development of the AskHERMES system is still at an early stage, and the knowledge resource is limited compared with Google or UpToDate. Nevertheless, the evaluation results show that AskHERMES' performance is comparable to the other systems. In particular, when answering complex clinical questions, it demonstrates the potential to outperform both Google and UpToDate systems.

Conclusions: AskHERMES, available at http://www.AskHERMES.org, has the potential to help physicians practice evidence-based medicine and improve the quality of patient care.

Citing Articles

Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy.

Aftab W, Apostolou Z, Bouazoune K, Straub T BMC Bioinformatics. 2024; 25(1):281.

PMID: 39192204 PMC: 11351623. DOI: 10.1186/s12859-024-05902-7.

Question answering systems for health professionals at the point of care-a systematic review.

Kell G, Roberts A, Umansky S, Qian L, Ferrari D, Soboczenski F J Am Med Inform Assoc. 2024; 31(4):1009-1024.

PMID: 38366879 PMC: 10990539. DOI: 10.1093/jamia/ocae015.

A transformer fine-tuning strategy for text dialect identification.

Humayun M, Yassin H, Shuja J, Alourani A, Abas P Neural Comput Appl. 2022; 35(8):6115-6124.

PMID: 36408287 PMC: 9665018. DOI: 10.1007/s00521-022-07944-5.

What Would it Take to get Biomedical QA Systems into Practice?.

Kell G, Marshall I, Wallace B, Jaun A Proc Conf Empir Methods Nat Lang Process. 2022; 2021:28-41.

PMID: 35663506 PMC: 9162079. DOI: 10.18653/v1/2021.mrqa-1.3.

BPI-MVQA: a bi-branch model for medical visual question answering.

Liu S, Zhang X, Zhou X, Yang J BMC Med Imaging. 2022; 22(1):79.

PMID: 35488285 PMC: 9052498. DOI: 10.1186/s12880-022-00800-x.

References

Cimino J, Aguirre A, Johnson S, Peng P . Generic queries for meeting clinical information needs. Bull Med Libr Assoc. 1993; 81(2):195-206. PMC: 225762. View

Goodyear-Smith F, Kerse N, Warren J, Arroll B . Evaluation of e-textbooks. DynaMed, MD Consult and UpToDate. Aust Fam Physician. 2008; 37(10):878-82. View

Ely J, Osheroff J, Ferguson K, Chambliss M, Vinson D, Moore J . Lifelong self-directed learning using a computer database of clinical questions. J Fam Pract. 1997; 45(5):382-8. View

Yu H, Lee M, Kaufman D, Ely J, Osheroff J, Hripcsak G . Development, implementation, and a cognitive evaluation of a definitional question answering system for physicians. J Biomed Inform. 2007; 40(3):236-51. DOI: 10.1016/j.jbi.2007.03.002. View

Griffiths K, Christensen H . Quality of web based information on treatment of depression: cross sectional survey. BMJ. 2000; 321(7275):1511-5. PMC: 27555. DOI: 10.1136/bmj.321.7275.1511. View

Collins S, Currie L, Bakken S, Cimino J . Information needs, Infobutton Manager use, and satisfaction by clinician type: a case study. J Am Med Inform Assoc. 2008; 16(1):140-2. PMC: 2605588. DOI: 10.1197/jamia.M2746. View

De Leo G, LeRouge C, Ceriani C, Niederman F . Websites most frequently used by physician for gathering medical information. AMIA Annu Symp Proc. 2007; :902. PMC: 1839616. View

Lee M, Cimino J, Zhu H, Sable C, Shanker V, Ely J . Beyond information retrieval--medical question answering. AMIA Annu Symp Proc. 2007; :469-73. PMC: 1839371. View

Cullen R . In search of evidence: family practitioners' use of the Internet for clinical information. J Med Libr Assoc. 2002; 90(4):370-9. PMC: 128953. View

10.

Cimino J, Borovtsov D . Leading a horse to water: using automated reminders to increase use of online decision support. AMIA Annu Symp Proc. 2008; :116-20. PMC: 2656005. View

11.

Muller H, Kenny E, Sternberg P . Textpresso: an ontology-based information retrieval and extraction system for biological literature. PLoS Biol. 2004; 2(11):e309. PMC: 517822. DOI: 10.1371/journal.pbio.0020309. View

12.

Cimino J, Li J, Bakken S, Patel V . Theoretical, empirical and practical approaches to resolving the unmet information needs of clinical information system users. Proc AMIA Symp. 2002; :170-4. PMC: 2244413. View

13.

Yu H, Cao Y . Using the weighted keyword model to improve information retrieval for answering biomedical questions. Summit Transl Bioinform. 2011; 2009:143-7. PMC: 3041568. View

14.

Wyatt J . Commentary: measuring quality and impact of the World Wide Web. BMJ. 1997; 314(7098):1879-81. PMC: 2127007. DOI: 10.1136/bmj.314.7098.1879. View

15.

Aronson A . Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2002; :17-21. PMC: 2243666. View

16.

Hoogendam A, Stalenhoef A, de Vries Robbe P, Overbeke A . Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res. 2008; 10(4):e29. PMC: 2629368. DOI: 10.2196/jmir.1012. View

17.

Cimino J, Elhanan G, Zeng Q . Supporting infobuttons with terminological knowledge. Proc AMIA Annu Fall Symp. 1997; :528-32. PMC: 2233544. View

18.

Elhadad N, Kan M, Klavans J, McKeown K . Customization in a unified framework for summarizing medical literature. Artif Intell Med. 2005; 33(2):179-98. DOI: 10.1016/j.artmed.2004.07.018. View

19.

Jadad A, Gagliardi A . Rating health information on the Internet: navigating to knowledge or to Babel?. JAMA. 1998; 279(8):611-4. DOI: 10.1001/jama.279.8.611. View

20.

Freeman M, Lauderdale S, Kendrach M, Woolley T . Google Scholar versus PubMed in locating primary literature to answer drug-related questions. Ann Pharmacother. 2009; 43(3):478-84. DOI: 10.1345/aph.1L223. View