Evaluating Casama: Contextualized Semantic Maps for Summarization of Lung Cancer Studies

Overview

Journal Comput Biol Med

Publisher Elsevier

Specialties Biology
General Medicine
Medical Informatics

Date 2017 Nov 18

PMID 29149658

Authors

Jean I Garcia-Gathright

Nicholas J Matiasz

Carlos Adame

Karthik V Sarma

Lauren Sauer

Nova F Smedley

Marshall L Spiegel

Jennifer Strunck

Edward B Garon

Ricky K Taira

Denise R Aberle

Alex A T Bui

Affiliations

Soon will be listed here.

Abstract

Objective: It is crucial for clinicians to stay up to date on current literature in order to apply recent evidence to clinical decision making. Automatic summarization systems can help clinicians quickly view an aggregated summary of literature on a topic. Casama, a representation and summarization system based on "contextualized semantic maps," captures the findings of biomedical studies as well as the contexts associated with patient population and study design. This paper presents a user-oriented evaluation of Casama in comparison to a context-free representation, SemRep.

Materials And Methods: The effectiveness of the representation was evaluated by presenting users with manually annotated Casama and SemRep summaries of ten articles on driver mutations in cancer. Automatic annotations were evaluated on a collection of articles on EGFR mutation in lung cancer. Seven users completed a questionnaire rating the summarization quality for various topics and applications.

Results: Casama had higher median scores than SemRep for the majority of the topics (p≤ 0.00032), all of the applications (p≤ 0.00089), and in overall summarization quality (p≤ 1.5e-05). Casama's manual annotations outperformed Casama's automatic annotations (p = 0.00061).

Discussion: Casama performed particularly well in the representation of strength of evidence, which was highly rated both quantitatively and qualitatively. Users noted that Casama's less granular, more targeted representation improved usability compared to SemRep.

Conclusion: This evaluation demonstrated the benefits of a contextualized representation for summarizing biomedical literature on cancer. Iteration on specific areas of Casama's representation, further development of its algorithms, and a clinically-oriented evaluation are warranted.

References

Novello S, Barlesi F, Califano R, Cufer T, Ekman S, Giaj Levra M . Metastatic non-small-cell lung cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2016; 27(suppl 5):v1-v27. DOI: 10.1093/annonc/mdw326. View

He Y, Sarntivijai S, Lin Y, Xiang Z, Guo A, Zhang S . OAE: The Ontology of Adverse Events. J Biomed Semantics. 2014; 5:29. PMC: 4120740. DOI: 10.1186/2041-1480-5-29. View

Sakaeda T, Tamon A, Kadoyama K, Okuno Y . Data mining of the public version of the FDA Adverse Event Reporting System. Int J Med Sci. 2013; 10(7):796-803. PMC: 3689877. DOI: 10.7150/ijms.6048. View

Kilicoglu H, Rosemblat G, Fiszman M, Rindflesch T . Constructing a semantic predication gold standard from the biomedical literature. BMC Bioinformatics. 2011; 12:486. PMC: 3281188. DOI: 10.1186/1471-2105-12-486. View

Garcia-Gathright J, Oh A, Abarca P, Han M, Sago W, Spiegel M . Representing and extracting lung cancer study metadata: study objective and study design. Comput Biol Med. 2015; 58:63-72. PMC: 4331232. DOI: 10.1016/j.compbiomed.2015.01.004. View

Ellison G, Zhu G, Moulis A, Dearden S, Speake G, McCormack R . EGFR mutation testing in lung cancer: a review of available methods and their use for analysis of tumour tissue and cytology samples. J Clin Pathol. 2012; 66(2):79-89. PMC: 3582044. DOI: 10.1136/jclinpath-2012-201194. View

Bundschus M, Dejori M, Stetter M, Tresp V, Kriegel H . Extraction of semantic biomedical relations from text using conditional random fields. BMC Bioinformatics. 2008; 9:207. PMC: 2386138. DOI: 10.1186/1471-2105-9-207. View

Hua L, Quan C . A Shortest Dependency Path Based Convolutional Neural Network for Protein-Protein Relation Extraction. Biomed Res Int. 2016; 2016:8479587. PMC: 4963603. DOI: 10.1155/2016/8479587. View

Blaschke C, Andrade M, Ouzounis C, Valencia A . Automatic extraction of biological information from scientific text: protein-protein interactions. Proc Int Conf Intell Syst Mol Biol. 2000; :60-7. View

10.

Ono T, Hishigaki H, Tanigami A, Takagi T . Automated extraction of information on protein-protein interactions from the biological literature. Bioinformatics. 2001; 17(2):155-61. DOI: 10.1093/bioinformatics/17.2.155. View

11.

Kushniruk A, McKeown K, Klavans J, Jordan D, Laflamme M, Patel V . Usability evaluation of an experimental text summarization system and three search engines: implications for the reengineering of health care interfaces. Proc AMIA Symp. 2002; :420-4. PMC: 2244493. View

12.

Hensing T, Salgia R . Molecular biomarkers for future screening of lung cancer. J Surg Oncol. 2013; 108(5):327-33. DOI: 10.1002/jso.23382. View

13.

Donnelly K . SNOMED-CT: The advanced terminology and coding system for eHealth. Stud Health Technol Inform. 2006; 121:279-90. View

14.

Elhadad N, Kan M, Klavans J, McKeown K . Customization in a unified framework for summarizing medical literature. Artif Intell Med. 2005; 33(2):179-98. DOI: 10.1016/j.artmed.2004.07.018. View

15.

Garcia-Gathright J, Matiasz N, Garon E, Aberle D, Taira R, Bui A . Toward patient-tailored summarization of lung cancer literature. IEEE EMBS Int Conf Biomed Health Inform. 2017; 2016:449-452. PMC: 5511748. DOI: 10.1109/BHI.2016.7455931. View

16.

Cao Y, Liu F, Simpson P, Antieau L, Bennett A, Cimino J . AskHERMES: An online question answering system for complex clinical questions. J Biomed Inform. 2011; 44(2):277-88. PMC: 3433744. DOI: 10.1016/j.jbi.2011.01.004. View

17.

Sahu S, Anand A . Drug-drug interaction extraction from biomedical texts using long short-term memory network. J Biomed Inform. 2018; 86:15-24. DOI: 10.1016/j.jbi.2018.08.005. View

18.

Sam L, Mendonca E, Li J, Blake J, Friedman C, Lussier Y . PhenoGO: an integrated resource for the multiscale mining of clinical and biological data. BMC Bioinformatics. 2009; 10 Suppl 2:S8. PMC: 2646241. DOI: 10.1186/1471-2105-10-S2-S8. View

19.

Bodenreider O . The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2003; 32(Database issue):D267-70. PMC: 308795. DOI: 10.1093/nar/gkh061. View

20.

Fundel K, Kuffner R, Zimmer R . RelEx--relation extraction using dependency parse trees. Bioinformatics. 2006; 23(3):365-71. DOI: 10.1093/bioinformatics/btl616. View