Deep learning-based large-scale named entity recognition for anatomical region of mammalian brain

Abstract

Background: Images of anatomical regions and neuron type distribution, as well as their related literature are valuable assets for neuroscience research. They are vital evidence and vehicles in discovering new phenomena and knowledge refinement through image and text big data. The knowledge acquired from image data generally echoes with the literature accumulated over the years. The knowledge within the literature can provide a comprehensive context for a deeper understanding of the image data. However, it is quite a challenge to manually identify the related literature and summarize the neuroscience knowledge in the large-scale corpus. Thus, neuroscientists are in dire need of an automated method to extract neuroscience knowledge from large-scale literature.

Methods: A proposed deep learning model named BioBERT-CRF extracts brain region entities from the WhiteText dataset. This model takes advantage of BioBERT and CRF to predict entity labels while training.

Results: The proposed deep learning model demonstrated comparable performance against or even outperforms the previous models on the WhiteText dataset. The BioBERT-CRF model has achieved the best average precision, recall, and F1 score of 81.3%, 84.0%, and 82.6%, respectively. We used the BioBERT-CRF model to predict brain region entities in a large-scale PubMed abstract dataset and used a rule-based method to normalize all brain region entities to three neuroscience dictionaries.

Conclusions: Our work shows that the BioBERT-CRF model can be well-suited for brain region entity extraction. The rankings of different brain region entities by their appearance in the large-scale corpus indicate the anatomical regions that researchers are most concerned about.

Graphical abstract

Keywords

brain region / entity extraction / literature mining / WhiteText / deep learning

References

[1] Sun,L., Patel,R., Liu,J., Chen,K., Wu,T., Li,J. ( 2009). Mining brain region connectivity for Alzheimer’s disease study via sparse inverse covariance estimation. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1335– 1344

[2] Sang,E. F. ( 2003). Introduction to the CoNLL -2003 shared task: Language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142– 147

[3] Wang,X., Zhang,Y., Ren,X., Zhang,Y., Zitnik,M., Shang,J., Langlotz,C. ( 2019). Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics, 35 : 1745– 1752

[4] Yoon,W., So,C. H., Lee,J. ( 2019). CollaboNet: collaboration of deep neural networks for biomedical named entity recognition. BMC Bioinformatics, 20 : 249

[5] Cho,M., Ha,J., Park,C. ( 2020). Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition. J. Biomed. Inform., 103 : 103381

[6] raj Kanakarajan,K., Kundumani,B. ( 2021). Bioelectra: Pretrained biomedical text encoder using discriminators. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 143– 154

[7] Leaman,R., an,R. ( 2013). DNorm: disease name normalization with pairwise learning to rank. Bioinformatics, 29 : 2909– 2917

[8] Dang,T. H., Le,H. Q., Nguyen,T. M. Vu,S. ( 2018). D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics, 34 : 3539– 3546

[9] Peng,Y., Chen,Q. ( 2020). An empirical study of multi-task learning on bert for biomedical text mining. In: Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, pp. 205– 214

[10] Leaman,R. ( 2016). TaggerOne: joint named entity recognition and normalization with semi-Markov models. Bioinformatics, 32 : 2839– 2846

[11] Wei,C. H., Harris,B. R., Kao,H. Y. ( 2013). tmVar: a text mining approach for extracting sequence variants in biomedical literature. Bioinformatics, 29 : 1433– 1439

[12] Wei,C. H., Phan,L., Feltz,J., Maiti,R., Hefferon,T. ( 2018). tmVar 2. 0: integrating genomic variant information from literature with dbSNP and ClinVar for precision medicine. Bioinformatics, 34 : 80– 87

[13] Wei,C. H., Kao,H. Y. ( 2012). SR4GN: a species recognition software tool for gene normalization. PLoS One, 7 : e38460

[14] Gu,Y., Tinn,R., Cheng,H., Lucas,M., Usuyama,N., Liu,X. ( 2020). Domain-specific language model pretraining for biomedical natural language processing. arXiv, 200715779

[15] Giorgi,J. M. Bader,G. ( 2018). Transfer learning for biomedical named entity recognition with neural networks. Bioinformatics, 34 : 4087– 4094

[16] Devlin,J., Chang,M. W., Lee,K. ( 2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv, 181004805

[17] Lee,J., Yoon,W., Kim,S., Kim,D., Kim,S., So,C. H. ( 2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36 : 1234– 1240

[18] French,L., Lane,S., Xu,L. ( 2009). Automated recognition of brain region mentions in neuroscience literature. Front. Neuroinform., 3 : 29

[19] Lafferty,J., McCallum,A. Pereira,F. ( 2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282– 289

[20] Richardet,R., Chappelier,J. C., Telefont,M. ( 2015). Large-scale extraction of brain connectivity from the neuroscientific literature. Bioinformatics, 31 : 1640– 1647

[21] Shardlow,M., Ju,M., Li,M., Reilly,C., Iavarone,E., McNaught,J. ( 2019). A text mining pipeline using active and deep learning aimed at curating information in computational neuroscience. Neuroinformatics, 17 : 391– 406

[22] Oh,S. W., Harris,J. A., Ng,L., Winslow,B., Cain,N., Mihalas,S., Wang,Q., Lau,C., Kuan,L., Henry,A. M. . ( 2014). A mesoscale connectome of the mouse brain. Nature, 508 : 207– 214

[23] Bota,M. Swanson,L. ( 2008). Bams neuroanatomical ontology: Design and implementation. Front. Neuroinform., 2 : 2

[24] Bowden,D. M. Dubach,M. ( 2003). NeuroNames 2002. Neuroinformatics, 1 : 43– 59

[25] Pyysalo,S., Ginter,F., Moen,H., Salakoski,T. ( 2013). Distributional semantics resources for biomedical text processing. In: Proceedings of LBM, pp. 39– 44

[26] Mikolov,T., Chen,K., Corrado,G. ( 2013). Efficient estimation of word representations in vector space. arXiv, 13013781

[27] Wu,Y., Schuster,M., Chen,Z., Le,Q. V., Norouzi,M., Macherey,W. ( 2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv, 160908144

RIGHTS & PERMISSIONS

The Authors (2022). Published by Higher Education Press.