Biomedical Named Entity Recognition Using Deep Neural Networks with Contextual Information

Overview

Journal BMC Bioinformatics

Publisher Biomed Central

Specialty Biology

Date 2019 Dec 29

PMID 31881938

Citations 27

Authors

Hyejin Cho

Hyunju Lee

Affiliations

Soon will be listed here.

Abstract

Background: In biomedical text mining, named entity recognition (NER) is an important task used to extract information from biomedical articles. Previously proposed methods for NER are dictionary- or rule-based methods and machine learning approaches. However, these traditional approaches are heavily reliant on large-scale dictionaries, target-specific rules, or well-constructed corpora. These methods to NER have been superseded by the deep learning-based approach that is independent of hand-crafted features. However, although such methods of NER employ additional conditional random fields (CRF) to capture important correlations between neighboring labels, they often do not incorporate all the contextual information from text into the deep learning layers.

Results: We propose herein an NER system for biomedical entities by incorporating n-grams with bi-directional long short-term memory (BiLSTM) and CRF; this system is referred to as a contextual long short-term memory networks with CRF (CLSTM). We assess the CLSTM model on three corpora: the disease corpus of the National Center for Biotechnology Information (NCBI), the BioCreative II Gene Mention corpus (GM), and the BioCreative V Chemical Disease Relation corpus (CDR). Our framework was compared with several deep learning approaches, such as BiLSTM, BiLSTM with CRF, GRAM-CNN, and BERT. On the NCBI corpus, our model recorded an F-score of 85.68% for the NER of diseases, showing an improvement of 1.50% over previous methods. Moreover, although BERT used transfer learning by incorporating more than 2.5 billion words, our system showed similar performance with BERT with an F-scores of 81.44% for gene NER on the GM corpus and a outperformed F-score of 86.44% for the NER of chemicals and diseases on the CDR corpus. We conclude that our method significantly improves performance on biomedical NER tasks.

Conclusion: The proposed approach is robust in recognizing biological entities in text.

Citing Articles

Construction, evaluation, and application of an electronic medical record corpus for cerebral palsy rehabilitation.

Xiao M, Pang Q, Zhu Y, Shuai L, Jin G Digit Health. 2024; 10:20552076241286260.

PMID: 39347507 PMC: 11437554. DOI: 10.1177/20552076241286260.

Utility analysis and demonstration of real-world clinical texts: A case study on Japanese cancer-related EHRs.

Yada S, Nishiyama T, Wakamiya S, Kawazoe Y, Imai S, Hori S PLoS One. 2024; 19(9):e0310432.

PMID: 39259727 PMC: 11389901. DOI: 10.1371/journal.pone.0310432.

Application of machine reading comprehension techniques for named entity recognition in materials science.

Huang Z, He L, Yang Y, Li A, Zhang Z, Wu S J Cheminform. 2024; 16(1):76.

PMID: 38956728 PMC: 11220966. DOI: 10.1186/s13321-024-00874-5.

Advancing entity recognition in biomedicine via instruction tuning of large language models.

Keloth V, Hu Y, Xie Q, Peng X, Wang Y, Zheng A Bioinformatics. 2024; 40(4).

PMID: 38514400 PMC: 11001490. DOI: 10.1093/bioinformatics/btae163.

A Combined Manual Annotation and Deep-Learning Natural Language Processing Study on Accurate Entity Extraction in Hereditary Disease Related Biomedical Literature.

Huang D, Zeng Q, Xiong Y, Liu S, Pang C, Xia M Interdiscip Sci. 2024; 16(2):333-344.

PMID: 38340264 PMC: 11289304. DOI: 10.1007/s12539-024-00605-2.

References

Li J, Sun Y, Johnson R, Sciaky D, Wei C, Leaman R . BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database (Oxford). 2016; 2016. PMC: 4860626. DOI: 10.1093/database/baw068. View

Galea D, Laponogov I, Veselkov K . Exploiting and assessing multi-source data for supervised biomedical named entity recognition. Bioinformatics. 2018; 34(14):2474-2482. PMC: 6041968. DOI: 10.1093/bioinformatics/bty152. View

Zhao Z, Yang Z, Luo L, Wang L, Zhang Y, Lin H . Disease named entity recognition from biomedical literature using a novel convolutional neural network. BMC Med Genomics. 2018; 10(Suppl 5):73. PMC: 5751782. DOI: 10.1186/s12920-017-0316-8. View

Huang C, Lu Z . Community challenges in biomedical text mining over 10 years: success, failure and the future. Brief Bioinform. 2015; 17(1):132-44. PMC: 4719069. DOI: 10.1093/bib/bbv024. View

Cho H, Choi W, Lee H . A method for named entity normalization in biomedical articles: application to diseases and plants. BMC Bioinformatics. 2017; 18(1):451. PMC: 5640957. DOI: 10.1186/s12859-017-1857-8. View

LeCun Y, Bengio Y, Hinton G . Deep learning. Nature. 2015; 521(7553):436-44. DOI: 10.1038/nature14539. View

Islamaj Dogan R, Leaman R, Lu Z . NCBI disease corpus: a resource for disease name recognition and concept normalization. J Biomed Inform. 2014; 47:1-10. PMC: 3951655. DOI: 10.1016/j.jbi.2013.12.006. View

Wang X, Zhang Y, Ren X, Zhang Y, Zitnik M, Shang J . Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics. 2018; 35(10):1745-1752. DOI: 10.1093/bioinformatics/bty869. View

Smith L, Tanabe L, Ando R, Kuo C, Chung I, Hsu C . Overview of BioCreative II gene mention recognition. Genome Biol. 2008; 9 Suppl 2:S2. PMC: 2559986. DOI: 10.1186/gb-2008-9-s2-s2. View

10.

Rocktaschel T, Weidlich M, Leser U . ChemSpot: a hybrid system for chemical named entity recognition. Bioinformatics. 2012; 28(12):1633-40. DOI: 10.1093/bioinformatics/bts183. View

11.

Blake C, Rindflesch T . Leveraging syntax to better capture the semantics of elliptical coordinated compound noun phrases. J Biomed Inform. 2017; 72:120-131. DOI: 10.1016/j.jbi.2017.07.001. View

12.

Gridach M . Character-level neural network for biomedical named entity recognition. J Biomed Inform. 2017; 70:85-91. DOI: 10.1016/j.jbi.2017.05.002. View

13.

Hochreiter S, Schmidhuber J . Long short-term memory. Neural Comput. 1997; 9(8):1735-80. DOI: 10.1162/neco.1997.9.8.1735. View

14.

Leaman R, Islamaj Dogan R, Lu Z . DNorm: disease name normalization with pairwise learning to rank. Bioinformatics. 2013; 29(22):2909-17. PMC: 3810844. DOI: 10.1093/bioinformatics/btt474. View

15.

Luo L, Yang Z, Yang P, Zhang Y, Wang L, Lin H . An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics. 2017; 34(8):1381-1388. DOI: 10.1093/bioinformatics/btx761. View

16.

Giorgi J, Bader G . Transfer learning for biomedical named entity recognition with neural networks. Bioinformatics. 2018; 34(23):4087-4094. PMC: 6247938. DOI: 10.1093/bioinformatics/bty449. View

17.

Bengio Y, Simard P, Frasconi P . Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw. 1994; 5(2):157-66. DOI: 10.1109/72.279181. View

18.

Graves A, Schmidhuber J . Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 2005; 18(5-6):602-10. DOI: 10.1016/j.neunet.2005.06.042. View

19.

Zhu Q, Li X, Conesa A, Pereira C . GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics. 2017; 34(9):1547-1554. PMC: 5925775. DOI: 10.1093/bioinformatics/btx815. View

20.

Habibi M, Weber L, Neves M, Wiegandt D, Leser U . Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics. 2017; 33(14):i37-i48. PMC: 5870729. DOI: 10.1093/bioinformatics/btx228. View