Classification of optical coherence tomography images using deep machine-learning methods

Abstract

BACKGROUND: Optical coherence tomography is a modern high-tech, insightful approach to detecting pathologies of the retina and preretinal layers of the vitreous body. However, the description and interpretation of study findings require advanced qualifications and special training of ophthalmologists and are highly time-consuming for both the doctor and the patient. Moreover, mathematical models based on artificial neural networks now allow for the automation of many image processing tasks. Therefore, addressing the issues of automated classification of optical coherence tomography images using deep learning artificial neural network models is crucial.

AIM: To develop architectures of mathematical (computer) models based on deep learning of convolutional neural networks for the classification of retinal optical coherence tomography images; to compare the results of computational experiments conducted using Python tools in Google Colaboratory with single-model and multimodel approaches, and evaluate classification accuracy; and to determine the optimal architecture of models based on artificial neural networks, as well as the values of the hyperparameters used.

MATERIALS AND METHODS: The original dataset included >2,000 anonymized optical coherence tomography images of real patients, obtained directly from the device with a resolution of 1,920×969×24 BPP. The number of image classes was 12. To create the training and validation datasets, a subject area of 1,100×550×24 BPP was “cut out”. Various approaches were studied: the possibility of using pretrained convolutional neural networks with transfer learning, techniques for resizing and augmenting images, and various combinations of the hyperparameters of models based on artificial neural networks. When compiling a model, the following parameters were used: Adam optimizer, categorical_crossentropy loss function, and accuracy. All technological operations involving images and models based on artificial neural networks were performed using Python language tools in Google Colaboratory.

RESULTS: Single-model and multimodel approaches to the classification of retinal optical coherence tomography images were developed. Computational experiments on the automated classification of such images obtained from a DRI OCT Triton tomograph using various architectures of models based on artificial neural networks showed an accuracy of 98–100% during training and validation, and 85% during an additional test, which is a satisfactory result. The optimal architecture of the model based on an artificial neural network, a six-layer convolutional network, was selected, and the values of its hyperparameters were determined.

CONCLUSION: Deep training of convolutional neural network models with various architectures, as well as their validation and testing, resulted in satisfactory classification accuracy of retinal optical coherence tomography images. These findings can be used in decision support systems in ophthalmology.

Keywords

artificial intelligence / medical data / dataset / machine learning / convolutional neural networks / optical coherence tomography

References

[1] Daker DS, Vekhid NK, Goldman DR, editors. Optical coherence tomography of the retina. Moscow: MEDpress-inform; 2021. (In Russ).

[2] Оптическая когерентная томография сетчатки / под ред. Дж.С. Дакера, Н.К. Вэхид, Д.Р. Голдмана. Москва : МЕДпресс-информ, 2021.

[3] Daker DS, Vekhid NK, Goldman DR, editors. Optical coherence tomography of the retina. Moscow: MEDpress-inform; 2021. (In Russ).

[4] Oakden-Rayner L, Palme LJ. Artificial intelligence in medicine: Validation and study design. In: Ranschart E, Morozov S, Algra P, editors. Artificial intelligence in medical imaging. Cham: Springer; 2019. Р:83–104. doi: 10.1007/978-3-319-94878-2_8

[5] Oakden-Rayner L., Palme L.J. Artificial intelligence in medicine: Validation and study design. In: Ranschart E., Morozov S., Algra P., editors. Artificial intelligence in medical imaging. Cham : Springer, 2019. Р. 83–104. doi: 10.1007/978-3-319-94878-2_8

[6] Oakden-Rayner L, Palme LJ. Artificial intelligence in medicine: Validation and study design. In: Ranschart E, Morozov S, Algra P, editors. Artificial intelligence in medical imaging. Cham: Springer; 2019. Р:83–104. doi: 10.1007/978-3-319-94878-2_8

[7] Ramsundar B, Istman P, Uolters P, Pande V. Deep learning in biology and medicine. Moscow: DMK Press; 2020. (In Russ).

[8] Рамсундар Б., Истман П., Уолтерс П., Панде В. Глубокое обучение в биологии и медицине. Москва : ДМК Пресс, 2020.

[9] Ramsundar B, Istman P, Uolters P, Pande V. Deep learning in biology and medicine. Moscow: DMK Press; 2020. (In Russ).

[10] Buduma N, Lokasho N. Foundations of deep learning. Creating Algorithms for Next Generation Artificial Intelligence. Moscow: Mann, Ivanov i Ferber; 2020. (In Russ).

[11] Будума Н., Локашо Н. Основы глубокого обучения. Создание алгоритмов для искусственного интеллекта следующего поколения. Москва : Манн, Иванов и Фербер, 2020.

[12] Buduma N, Lokasho N. Foundations of deep learning. Creating Algorithms for Next Generation Artificial Intelligence. Moscow: Mann, Ivanov i Ferber; 2020. (In Russ).

[13] Foster D. Generative deep learning. Creative potential of neural networks. Saint Petersburg: Piter; 2020. (In Russ).

[14] Фостер Д. Генеративное глубокое обучение. Творческий потенциал нейронных сетей. Санкт-Петербург : Питер, 2020.

[15] Foster D. Generative deep learning. Creative potential of neural networks. Saint Petersburg: Piter; 2020. (In Russ).

[16] Postolit AV. Fundamentals of Artificial Intelligence in Python examples. Saint Petersburg: BKhV-Peterburg; 2021. (In Russ).

[17] Постолит А.В. Основы искусственного интеллекта в примерах на Python. Санкт-Петербург : БХВ-Петербург, 2021.

[18] Postolit AV. Fundamentals of Artificial Intelligence in Python examples. Saint Petersburg: BKhV-Peterburg; 2021. (In Russ).

[19] Arzamastsev AA, Fabrikantov OL, Zenkova NA, Kulagina EV. Software development for analysing the optical coherence tomography protocols of the retina and automatic composition of their descriptions. Sovremennye problemy nauki i obrazovaniya. 2021;(6). EDN: PCVMRX doi: 10.17513/spno.31208

[20] Арзамасцев А.А., Фабрикантов О.Л., Зенкова Н.А., Кулагина Е.В. Разработка программного обеспечения для анализа протоколов оптической когерентной томографии сетчатки глаза и автоматизированного составления их описаний // Современные проблемы науки и образования. 2021. № 6. EDN: PCVMRX doi: 10.17513/spno.31208

[21] Arzamastsev AA, Fabrikantov OL, Zenkova NA, Kulagina EV. Software development for analysing the optical coherence tomography protocols of the retina and automatic composition of their descriptions. Sovremennye problemy nauki i obrazovaniya. 2021;(6). EDN: PCVMRX doi: 10.17513/spno.31208

[22] Vasiliev YA, Vlazimirsky AV, Omelyanskaya OV, et al. Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics. Digital Diagnostics. 2023;4(3):252−267. doi: 10.17816/DD321971

[23] Васильев Ю.А., Владзимирский А.В., Омелянская О.В., и др. Методология тестирования и мониторинга программного обеспечения на основе технологий искусственного интеллекта для медицинской диагностики // Digital Diagnostics. 2023. Т. 4, № 3. С. 252−267. doi: 10.17816/DD321971

[24] Vasiliev YA, Vlazimirsky AV, Omelyanskaya OV, et al. Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics. Digital Diagnostics. 2023;4(3):252−267. doi: 10.17816/DD321971

[25] Katalevskaya EA, Katalevsky DYu, Tyurikov MI, Shaykhutdinova EF, Sizov AYu. Algorithm for segmentation of visual signs of diabetic retinopathy (DR) and diabetic macular edema (DME) in digital fundus images. Russian Journal of Telemedicine and e-health. 2021;7(4):17–26. EDN: PPSPAL doi: 10.29188/2712-9217-2021-7-4-17-26

[26] Каталевская Е.А., Каталевский Д.Ю., Тюриков М.И., Шайхутдинова Э.Ф., Сизов А.Ю. Алгоритм сегментации визуальных признаков диабетической ретинопатии (ДР) и диабетического макулярного отёка (ДМО) на цифровых фотографиях глазного дна // Российский журнал телемедицины и электронного здравоохранения. 2021. Т. 7, № 4. С. 17–26. EDN: PPSPAL doi: 10.29188/2712-9217-2021-7-4-17-26

[27] Katalevskaya EA, Katalevsky DYu, Tyurikov MI, Shaykhutdinova EF, Sizov AYu. Algorithm for segmentation of visual signs of diabetic retinopathy (DR) and diabetic macular edema (DME) in digital fundus images. Russian Journal of Telemedicine and e-health. 2021;7(4):17–26. EDN: PPSPAL doi: 10.29188/2712-9217-2021-7-4-17-26

[28] Kepp T, Sudkamp H, Burchard C, et al. Segmentation of retinal low-cost optical coherence tomography images using deep learning. Medical Imaging 2020: Computer-Aided Diagnosis. 2020;11314:389–396. doi: 10.48550/arXiv.2001.08480

[29] Kepp T., Sudkamp H., Burchard C., et al. Segmentation of retinal low-cost optical coherence tomography images using deep learning // Medical Imaging 2020: Computer-Aided Diagnosis. 2020. Vol. 11314. P. 389–396. doi: 10.48550/arXiv.2001.08480

[30] Kepp T, Sudkamp H, Burchard C, et al. Segmentation of retinal low-cost optical coherence tomography images using deep learning. Medical Imaging 2020: Computer-Aided Diagnosis. 2020;11314:389–396. doi: 10.48550/arXiv.2001.08480

[31] Sakhnov SN, Axenov KD, Axenova LE, et al. Development of a cataract screening model using an open dataset and deep machine learning algorithms. Fyodorov Journal of Ophthalmic Surgery. 2022;(4S):13–20. EDN: VEGPAW doi: 10.25276/0235-4160-2022-4S-13-20

[32] Сахнов С.Н., Аксенов К.Д., Аксенова Л.Е., и др. Разработка модели скрининга катаракты с использованием открытого набора данных и алгоритмов глубокого машинного обучения // Офтальмохирургия. 2022. № 4S. С. 13–20. EDN: VEGPAW doi: 10.25276/0235-4160-2022-4S-13-20

[33] Sakhnov SN, Axenov KD, Axenova LE, et al. Development of a cataract screening model using an open dataset and deep machine learning algorithms. Fyodorov Journal of Ophthalmic Surgery. 2022;(4S):13–20. EDN: VEGPAW doi: 10.25276/0235-4160-2022-4S-13-20

[34] Shukhaev SV, Mordovtseva EA, Pustozerov EA, Kudlakhmedov SS. Application of convolutional neural networks to define Fuchs endothelial dystrophy. Fyodorov Journal of Ophthalmic Surgery. 2022;(4S):70–76. EDN: WEZTKV doi: 10.25276/0235-4160-2022-4S-70-76

[35] Шухаев С.В., Мордовцева Е.А., Пустозеров Е.А., Кудлахмедов Ш.Ш. Применение сверточных нейронных сетей для определения эндотелиальной дистрофии Фукса // Офтальмохирургия. 2022. № 4S. С. 70–76. EDN: WEZTKV doi: 10.25276/0235-4160-2022-4S-70-76

[36] Shukhaev SV, Mordovtseva EA, Pustozerov EA, Kudlakhmedov SS. Application of convolutional neural networks to define Fuchs endothelial dystrophy. Fyodorov Journal of Ophthalmic Surgery. 2022;(4S):70–76. EDN: WEZTKV doi: 10.25276/0235-4160-2022-4S-70-76

RIGHTS & PERMISSIONS

Eco-Vector