Automatic diagnosis of multiple fundus lesions based on depth graph neural network

Abstract

Fundus images are commonly used to capture changes in fundus structures and the severity of fundus lesions, and are the basis for detecting and treating ophthalmic diseases as well as other important diseases. This study proposes an automatic diagnosis method for multiple fundus lesions based on a deep graph neural network (GNN). 2 083 fundus images were collected and annotated to develop and evaluate the performance of the algorithm. First, high-level semantic features of fundus images are extracted using deep convolutional neural networks (CNNs). Then the features are input into the GNN to model the correlation between different lesions by mining and learning the correlation between lesions. Finally, the input and output features of the GNN are fused, and a multi-label classifier is used to complete the automatic diagnosis of fundus lesions. Experimental results show that the method proposed in this study can learn the correlations between lesions to improve the diagnostic performance of the algorithm, achieving better performance than the original ResNet and DenseNet models in both qualitative and quantitative evaluation.

References

[1] YeH, ZhangQ, LiuX, et al.. Prevalence of age-related macular degeneration in an elderly urban chinese population in China: the Jiangning eye study[J]. Investigative ophthalmology & visual science, 2014, 55(10):6374-6380

[2] SongL, LinJ, WangZ J, et al.. An end-to-end multi-task deep learning framework for skin lesion analysis[J]. IEEE journal of biomedical and health informatics, 2020, 24(10):2912-2921

[3] OgurtsovaK, Da Rocha FernandesJ, HuangY, et al.. Idf diabetes atlas: global estimates for the prevalence of diabetes for 2015 and 2040[J]. Diabetes research and clinical practice, 2017, 128: 40-50

[4] MukaidaM, OkamiY, SuetakeN, et al.. Contour line extraction of vein with use of special tracking strategy and fuzzy inference for arteriosclerosis diagnosis of retinal blood vessel[C], 2020, New York, IEEE: 435-440

[5] LaiX, LiX, QianR, et al.. Four models for automatic recognition of left and right eye in fundus images[C], 2019, Berlin, Springer: 507-517

[6] AgarwalA, GuliaS, ChaudharyS, et al.. Automatic glaucoma detection using adaptive threshold based technique in fundus image[C], 2015, New York, IEEE: 416-420

[7] RathoreS, AswalA, SaranyaP. Bright lesion detection in retinal fundus images for diabetic retinopathy detection using machine learning approach[J]. Annals of the Romanian Society for Cell Biology, 2021, 25(5):4360-4367

[8] ColomerA, IgualJ, NaranjoV. Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images[J]. Sensors, 2020, 20(4):1005

[9] AlaguselviR, MuruganK. Performance analysis of automated lesion detection of diabetic retinopathy using morphological operation[J]. Signal, Image and Video Processing, 2021, 15(4): 797-805

[10] JadhavM, ShaikhM, SardarV. Automated microaneurysms detection in fundus images for early diagnosis of diabetic retinopathy[C], 2021, Singapore, Springer: 87-95

[11] LiZ, JiangJ, ChenK, et al.. Preventing corneal blindness caused by keratitis using artificial intelligence[J]. Nature communications, 2021, 12(1):1-12

[12] JiangJ, WangL, FuH, et al.. Automatic classification of heterogeneous slit-illumination images using an ensemble of cost-sensitive convolutional neural networks[J]. Annals of translational medicine, 2021, 9(7):550

[13] ImranA, LiJ, PeiY, et al.. Fundus image-based cataract classification using a hybrid convolutional and recurrent neural network[J]. The visual computer, 2021, 37(8): 2407-2417

[14] LiZ, GuoC, LinD, et al.. Deep learning for automated glaucomatous optic neuropathy detection from ultra-widefield fundus images[J]. British journal of ophthalmology, 2021, 105(11):1548-1554

[15] LiZ, QiangW, ChenH, et al.. Artificial intelligence to detect malignant eyelid tumors from photographic images[J]. NPJ digital medicine, 2022, 5(1):1-9

[16] GulshanV, PengL, CoramM, et al.. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs[J]. Jama, 2016, 316(22):2402-2410

[17] OhK, KangH M, LeemD, et al.. Early detection of diabetic retinopathy based on deep learning and ultra-wide-field fundus images[J]. Scientific reports, 2021, 11(1):1-9

[18] TangS, YuF. Construction and verification of retinal vessel segmentation algorithm for color fundus image under bp neural network model[J]. The journal of supercomputing, 2021, 77(4):3870-3884

[19] DasS, KharbandaK, SuchethaM, et al.. Deep learning architecture based on segmented fundus image features for classification of diabetic retinopathy[J]. Biomedical signal processing and control, 2021, 68: 102600

[20] RajA, ShahN A, TiwariA K. A novel approach for fundus image enhancement[J]. Biomedical signal processing and control, 2022, 71: 103208

[21] LiZ, JiangJ, ZhouH, et al.. Development of a deep learning-based image eligibility verification system for detecting and filtering out ineligible fundus images: a multicentre study[J]. International journal of medical informatics, 2021, 147: 104363

[22] ShenY, ShengB, FangR, et al.. Domain-invariant interpretable fundus image quality assessment[J]. Medical image analysis, 2020, 61: 101654

[23] WangJ, YangL, HuoZ, et al.. Multi-label classification of fundus images with efficientnet[J]. IEEE access, 2020, 8: 212499-212508

[24] GourN, KhannaP. Multi-class multi-label ophthalmological disease detection using transfer learning based convolutional neural network[J]. Biomedical signal processing and control, 2021, 66: 102329

[25] DingW, SunY, RenL, et al.. Multiple lesions detection of fundus images based on convolution neural network algorithm with improved sfla[J]. IEEE access, 2020, 8: 97618-97631

[26] PanX, JinK, CaoJ, et al.. Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning[J]. Graefe’s archive for clinical and experimental ophthalmology, 2020, 258(4):779-785

[27] LiC, YeJ, HeJ, et al.. Dense correlation network for automated multi-label ocular disease detection with paired color fundus photographs[C], 2020, New York, IEEE: 1-4

[28] HeK, ZhangX, RenS, et al.. Deep residual learning for image recognition[C], 2016, New York, IEEE: 770-778

[29] HuangG, LiuZ, Van Der MaatenL, et al.. Densely connected convolutional networks[C], 2017, New York, IEEE: 4700-4708

[30] MoridM A, BorjaliA, Del FiolG. A scoping review of transfer learning research on medical image analysis using imagenet[J]. Computers in biology and medicine, 2021, 128: 104115

[31] ChenT, XuM, HuiX, et al.. Learning semantic-specific graph representation for multi-label image recognition[C], 2019, New York, IEEE: 522-531

[32] DeyR, SalemF M. Gate-variants of gated recurrent unit (GRU) neural networks[C], 2017, New York, IEEE: 1597-1600

[33] LiZ Q, SunJ, WuX J, et al.. Multiplication fusion of sparse and collaborative-competitive representation for image classification[J]. International journal of machine learning and cybernetics, 2020, 11: 2357-2369