Neural network model development for detecting atypical mitoses in histological slides

Abstract

Background: Modern computer systems allow digitizing and examining images of histological preparations, which led the authors to the idea of using machine learning tools in digital pathohistology. The ability of neural networks to find sub-visual image features in digitized histological preparations provides the basis for better qualitative and quantitative image analysis. Existing machine learning methods provide good accuracy and speed in recognizing various images, which gives hope for their wide application, including in oncologic diagnostics.

AIM: Use methods of mathematical modeling to identify pathological mitoses in histological preparations as the main sign of the difference between malignant and benign tumor growth.

MATERIALS AND METHODS: Histological images of the N.N. Priorov National Medical Research Center of Traumatology and Orthopedics were used as a data set for the neural network model. The model was tested using 188 histologic slides from 67 patients treated at the institute. Histological preparations were scanned on a Leica Aperio CS2 microscope with a ×400 resolution and converted into JPEG format with further processing. Next, the test images were analyzed in streaming mode using the created neural network model in order to obtain the coordinates of the desired diagnostic object — pathological mitosis and the probability with which the model found the object of this category. The obtained images were analyzed by a pathologist to determine whether the detected object corresponded to pathological mitosis.

RESULTS: The authors have chosen an architecture, developed a methodology for training a neural network, and created a model that can be used to detect pathologic mitoses in histologic preparations. The authors do not attempt to replace the physician, but show the possibility of an integrated approach to data analysis by a computer system and a pathologist.

Conclusions: The developed mathematical model of neural network used as a part of technological solution for recognizing pathological mitoses in scanned histological preparations can be used as a tool to reduce the time of research and increase the accuracy of diagnosis by a pathologist.

Keywords

neural network / mathematical model / artificial intelligence / tumor / pathological mitosis / machine learning / bone pathology

References

[1] Dorfman HD, Czerniak B. Bone tumors. 2nd edition. St. Louis: Mosby; 2015. 1261 p.

[2] Dorfman H.D., Czerniak B. Bone tumors. 2nd edition. St. Louis: Mosby, 2015. 1261 p.

[3] Girshick R. Fast R-CNN. In: IEEE International Conference on Computer Vision (ICCV); 2015.

[4] Girshick R. Fast R-CNN. In: IEEE International Conference on Computer Vision (ICCV), 2015.

[5] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2017;9(6):1137–1149. doi: 10.1109/TPAMI.2016.2577031

[6] Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks // IEEE Trans Pattern Anal Mach Intell. 2017. Vol. 39, № 6. Р. 1137–1149. doi: 10.1109/TPAMI.2016.2577031

[7] Ren S, He K, Girshick R, Zhang X, Sun J. Object Detection Networks on Convolutional Feature Maps. IEEE Trans Pattern Anal Mach Intell. 2017;39(7):1476–1481. doi: 10.1109/TPAMI.2016.2601099

[8] Ren S., He K., Girshick R., Zhang X., Sun J. Object Detection Networks on Convolutional Feature Maps // IEEE Trans Pattern Anal Mach Intell. 2017. Vol. 39, № 7. Р. 1476–1481. doi: 10.1109/TPAMI.2016.2601099

[9] Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature Pyramid Networks for Object Detection, Computer Science > Computer Vision and Pattern Recognition [Submitted on 9 Dec 2016 (v1), last revised 19 Apr 2017 (this version, v2)]. Available from: https://arxiv.org/abs/1612.03144

[10] Lin T-Y., Dollár P., Girshick R., He K., Hariharan B., Belongie S. Feature Pyramid Networks for Object Detection, Computer Science > Computer Vision and Pattern Recognition [Submitted on 9 Dec 2016 (v1), last revised 19 Apr 2017 (this version, v2)]. Режим доступа: https://arxiv.org/abs/1612.03144

[11] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR; 2014. arXiv: 1311.2524.

[12] Girshick R., Donahue J., Darrell T., Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR, 2014. arXiv: 1311.2524.

[13] Girshick R, Donahue J, Darrell T, Malik J. Region-Based Convolutional Networks for Accurate Object Detection and Segmentation. IEEE Trans Pattern Anal Mach Intell. 2016;38(1):142–58. doi: 10.1109/TPAMI.2015.2437384

[14] Girshick R., Donahue J., Darrell T., Malik J. Region-Based Convolutional Networks for Accurate Object Detection and Segmentation // IEEE Trans Pattern Anal Mach Intell. 2016. Vol. 38, № 1. Р. 142–58. doi: 10.1109/TPAMI.2015.2437384

[15] Uijlings J, van de Sande K, Gevers T, Smeulders A. Selective search for object recognition. International Journal of Computer Vision. 2013;104(2):154–171. doi: 10.1007/s11263-013-0620-5

[16] Uijlings J., van de Sande K., Gevers T., Smeulders A. Selective search for object recognition // International Journal of Computer Vision. 2013. Vol. 104, № 2. P. 154–171. doi: 10.1007/s11263-013-0620-5

[17] He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. 2020;42(2):386–397. doi: 10.1109/TPAMI.2018.2844175

[18] He K., Gkioxari G., Dollar P., Girshick R., Mask R-CNN // IEEE Trans Pattern Anal Mach Intell. 2020. Vol. 42, № 2. Р. 386–397. doi: 10.1109/TPAMI.2018.2844175

[19] Detectron [Internet]. Available from: https://github.com/facebookresearch/Detectron

[20] Detectron [Интернет]. Режим доступа: https://github.com/facebookresearch/Detectron

[21] Pantanowitz L, Quiroga-Garza GM, BienRonen L, et al. An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study. Lancet Digital Health. 2020;2(8):e407–e416. doi: 10.1016/S2589-7500(20)30159-X

[22] Pantanowitz L., Quiroga-Garza G.M., BienRonen L., et al. An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study // Lancet Digital Health. 2020. Vol. 2, № 8. Р. e407–e416. doi: 10.1016/S2589-7500(20)30159-X

[23] Pantanowitz L, Hartman D, Yan Qi, Eun Yoon Cho, et al. Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses. Diagn Pathol. 2020;15(1):80. doi: 10.1186/s13000-020-00995-z

[24] Pantanowitz L., Hartman D., Yan Qi, Eun Yoon Cho, et al. Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses // Diagn Pathol. 2020. Vol. 15, № 1. Р. 80. doi: 10.1186/s13000-020-00995-z

[25] Van EttenIn A. Satellite Imagery Multiscale Rapid Detection with Windowed Networks. arXiv: 1809.09978v1. Available from: https://arxiv.org/pdf/1809.09978.pdf

[26] Van EttenIn A. Satellite Imagery Multiscale Rapid Detection with Windowed Networks. arXiv: 1809.09978v1. Режим доступа: https://arxiv.org/pdf/1809.09978.pdf

[27] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 10 Apr 2015. arXiv: 1409.1556. Available from: https://arxiv.org/pdf/1409.1556.pdf

[28] Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. 10 Apr 2015. arXiv: 1409.1556. Режим доступа: https://arxiv.org/pdf/1409.1556.pdf

[29] Barisonia L, Hodgin JB. Digital pathology in nephrology clinical trials, research, and pathology practice. Curr Opin Nephrol Hypertens. 2017;26(6):450–459. doi: 10.1097/MNH.0000000000000360

[30] Barisonia L., Hodgin J.B. Digital pathology in nephrology clinical trials, research, and pathology practice // Curr Opin Nephrol Hypertens. 2017. Vol. 26, № 6. Р. 450–459. doi: 10.1097/MNH.0000000000000360

[31] Burt JR, Torosdagli N, Khosravan N, et al. Deep learning beyond cats and dogs: recent advances in diagnosing breast cancer with deep neural networks. Br J Radiol. 2018;91(1089):20170545. doi: 10.1259/bjr.20170545

[32] Burt J.R., Torosdagli N., Khosravan N., et al. Deep learning beyond cats and dogs: recent advances in diagnosing breast cancer with deep neural networks // Br J Radiol. 2018. Vol. 91, № 1089. Р. 20170545. doi: 10.1259/bjr.20170545

[33] Mermel C, Kunal Nagpal MS. Using AI to identify the aggressiveness of prostate cancer. Google Health. 2020. Available from: https://blog.google/technology/health/using-ai-identify-aggressiveness-prostate-cancer

[34] Mermel C., Kunal Nagpal M.S. Using AI to identify the aggressiveness of prostate cancer // Google Health. 2020. Режим доступа: https://blog.google/technology/health/using-ai-identify-aggressiveness-prostate-cancer

[35] Nagpal K, Foote D, Tan F, et al. Development and Validation of a Deep Learning Algorithm for Gleason Grading of Prostate Cancer from Biopsy Specimens. JAMA Oncol. 2020;6(9):1–9. doi: 10.1001/jamaoncol.2020.2485

[36] Nagpal K., Foote D., Tan F., et al. Development and Validation of a Deep Learning Algorithm for Gleason Grading of Prostate Cancer from Biopsy Specimens // JAMA Oncol. 2020. Vol. 6, № 9. Р. 1–9. doi: 10.1001/jamaoncol.2020.2485

Funding

Госзадание по научно-исследовательской работе, РКState assignment for scientific research work, RK(124040100041-5)

RIGHTS & PERMISSIONS

Eco-Vector