Deep learning-based large-scale named entity recognition for anatomical region of mammalian brain

Abstract

Background: Images of anatomical regions and neuron type distribution, as well as their related literature are valuable assets for neuroscience research. They are vital evidence and vehicles in discovering new phenomena and knowledge refinement through image and text big data. The knowledge acquired from image data generally echoes with the literature accumulated over the years. The knowledge within the literature can provide a comprehensive context for a deeper understanding of the image data. However, it is quite a challenge to manually identify the related literature and summarize the neuroscience knowledge in the large-scale corpus. Thus, neuroscientists are in dire need of an automated method to extract neuroscience knowledge from large-scale literature.

Methods: A proposed deep learning model named BioBERT-CRF extracts brain region entities from the WhiteText dataset. This model takes advantage of BioBERT and CRF to predict entity labels while training.

Results: The proposed deep learning model demonstrated comparable performance against or even outperforms the previous models on the WhiteText dataset. The BioBERT-CRF model has achieved the best average precision, recall, and F1 score of 81.3%, 84.0%, and 82.6%, respectively. We used the BioBERT-CRF model to predict brain region entities in a large-scale PubMed abstract dataset and used a rule-based method to normalize all brain region entities to three neuroscience dictionaries.

Conclusions: Our work shows that the BioBERT-CRF model can be well-suited for brain region entity extraction. The rankings of different brain region entities by their appearance in the large-scale corpus indicate the anatomical regions that researchers are most concerned about.

Author summary

In this study, the BioBERT-CRF model was used to extract brain region entities from a large-scale PubMed abstract dataset and a normalization pipeline was created for normalizing all the labeled brain region entities extracted to three neuroscience dictionaries. Prior to entity prediction, the performance of the BioBERT-CRF model was evaluated using the WhiteText dataset. Compared to other deep learning models, the BioBERT-CRF model achieved the best average precision, recall, and F1 score of 81.3%, 84.0%, and 82.6%, respectively. Our work demonstrates how the BioBERT-CRF model can be well-suited for neuroscience brain region entity extraction. The rankings of different brain region entities by their appearance in the large-scale corpus reflect the anatomical regions that researchers are most concerned with.

Graphical abstract

Keywords

brain region / entity extraction / literature mining / WhiteText / deep learning

References

[1] Sun,L., Patel,R., Liu,J., Chen,K., Wu,T., Li,J. ( 2009). Mining brain region connectivity for Alzheimer’s disease study via sparse inverse covariance estimation. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1335– 1344

[2] Sang,E. F. ( 2003). Introduction to the CoNLL -2003 shared task: Language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142– 147

[3] Wang,X., Zhang,Y., Ren,X., Zhang,Y., Zitnik,M., Shang,J., Langlotz,C. ( 2019). Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics, 35 : 1745– 1752

[4] Yoon,W., So,C. H., Lee,J. ( 2019). CollaboNet: collaboration of deep neural networks for biomedical named entity recognition. BMC Bioinformatics, 20 : 249

[5] Cho,M., Ha,J., Park,C. ( 2020). Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition. J. Biomed. Inform., 103 : 103381

[6] raj Kanakarajan,K., Kundumani,B. ( 2021). Bioelectra: Pretrained biomedical text encoder using discriminators. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 143– 154

[7] Leaman,R., an,R. ( 2013). DNorm: disease name normalization with pairwise learning to rank. Bioinformatics, 29 : 2909– 2917

[8] Dang,T. H., Le,H. Q., Nguyen,T. M. Vu,S. ( 2018). D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics, 34 : 3539– 3546

[9] Peng,Y., Chen,Q. ( 2020). An empirical study of multi-task learning on bert for biomedical text mining. In: Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, pp. 205– 214

[10] Leaman,R. ( 2016). TaggerOne: joint named entity recognition and normalization with semi-Markov models. Bioinformatics, 32 : 2839– 2846

[11] Wei,C. H., Harris,B. R., Kao,H. Y. ( 2013). tmVar: a text mining approach for extracting sequence variants in biomedical literature. Bioinformatics, 29 : 1433– 1439

[12] Wei,C. H., Phan,L., Feltz,J., Maiti,R., Hefferon,T. ( 2018). tmVar 2. 0: integrating genomic variant information from literature with dbSNP and ClinVar for precision medicine. Bioinformatics, 34 : 80– 87

[13] Wei,C. H., Kao,H. Y. ( 2012). SR4GN: a species recognition software tool for gene normalization. PLoS One, 7 : e38460

[14] Gu,Y., Tinn,R., Cheng,H., Lucas,M., Usuyama,N., Liu,X. ( 2020). Domain-specific language model pretraining for biomedical natural language processing. arXiv, 200715779

[15] Giorgi,J. M. Bader,G. ( 2018). Transfer learning for biomedical named entity recognition with neural networks. Bioinformatics, 34 : 4087– 4094

[16] Devlin,J., Chang,M. W., Lee,K. ( 2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv, 181004805

[17] Lee,J., Yoon,W., Kim,S., Kim,D., Kim,S., So,C. H. ( 2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36 : 1234– 1240

[18] French,L., Lane,S., Xu,L. ( 2009). Automated recognition of brain region mentions in neuroscience literature. Front. Neuroinform., 3 : 29

[19] Lafferty,J., McCallum,A. Pereira,F. ( 2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282– 289

[20] Richardet,R., Chappelier,J. C., Telefont,M. ( 2015). Large-scale extraction of brain connectivity from the neuroscientific literature. Bioinformatics, 31 : 1640– 1647

[21] Shardlow,M., Ju,M., Li,M., Reilly,C., Iavarone,E., McNaught,J. ( 2019). A text mining pipeline using active and deep learning aimed at curating information in computational neuroscience. Neuroinformatics, 17 : 391– 406

[22] Oh,S. W., Harris,J. A., Ng,L., Winslow,B., Cain,N., Mihalas,S., Wang,Q., Lau,C., Kuan,L., Henry,A. M. . ( 2014). A mesoscale connectome of the mouse brain. Nature, 508 : 207– 214

[23] Bota,M. Swanson,L. ( 2008). Bams neuroanatomical ontology: Design and implementation. Front. Neuroinform., 2 : 2

[24] Bowden,D. M. Dubach,M. ( 2003). NeuroNames 2002. Neuroinformatics, 1 : 43– 59

[25] Pyysalo,S., Ginter,F., Moen,H., Salakoski,T. ( 2013). Distributional semantics resources for biomedical text processing. In: Proceedings of LBM, pp. 39– 44

[26] Mikolov,T., Chen,K., Corrado,G. ( 2013). Efficient estimation of word representations in vector space. arXiv, 13013781

[27] Wu,Y., Schuster,M., Chen,Z., Le,Q. V., Norouzi,M., Macherey,W. ( 2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv, 160908144

ACKNOWLEDGEMENTS

We thank the MOST group members of Britton Chance Center for Biomedical Photonics for assistance with experiments and comments on the manuscript. We also thank Nannan Li for the helpful discussions. This work was supported by the National Science and Technology Innovation 2030 Grant (No. 2021ZD0201002), the National Natural Science Foundation of China (Nos. T2122015 and 61890954), CAMS Innovation Fund for Medical Sciences (No. 2019-I2M-5-014) and Suzhou Prospective Application Research Project (No. SYG201915).

COMPLIANCE WITH ETHICS GUIDELINES

The authors Xiaokang Chai, Yachao Di, Zhao Feng, Yue Guan, Guoqing Zhang, Anan Li and Qingming Luo declare they have no conflict of interests.

This article does not contain any studies with human or animal subjects performed by any of the authors.

OPEN ACCESS

This article is licensed by the CC By under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2022 The Authors (2022). Published by Higher Education Press.