Overview of modern digital diagnostic image markup tools

Abstract

Background. In modern medicine, artificial intelligence algorithms are being actively introduced, for testing and training of which a large amount of labeled datasets is required. Software for labeling (annotation) of digital diagnostic images is a necessary element when creating datasets.

Aim. To review the capabilities and comparative analysis of the functionality of the most common available software for annotating digital diagnostic images.

Material and methods. Five free and one commercial software product for annotation of digital diagnostic images participated in the comparative analysis. When testing the marking process on medical images for several target types of pathology, the usability of the graphical user interface and functionality was evaluated. The functionality of the software products has been tested by radiologists with over 5 years of experience. In addition, a review of semi-automatic segmentation methods implemented in the studied software products was carried out. As initial medical images, datasets of computed tomography studies obtained from open sources, were used.

Results. Comparison of software functionality for annotation of digital diagnostic images was made: supported formats; loading, presenting and saving original images and annotation data; the possibility of visualization of medical images; annotation tools. The algorithms underlying semi-automatic segmentation methods were studied and systematized. The requirements for the basic functionality of software for labeling digital diagnostic images have been formulated. The results obtained create a systematic basis for developing recommendations for radiologists on the choice and use of digital diagnostic image marking tools.

Conclusion. The most complete functionality in the field of segmentation of digital diagnostic images among the considered free software has 3D Slicer; in the case of annotation for detection tasks, it is convenient to use the Supervisely, CVAT platforms; for automatic segmentation of some types of pathology and organs, 3D Slicer extensions and ready-made models in Medseg can be used.

Keywords

labeling of digital diagnostic images / segmentation / 3D Slicer / MITK / ITK-SNAP / Medseg / CVAT / Supervisely

References

[1] Meess KM, Izzo RL, Dryjski ML, Curl RE, Harris LM, Springer M, Siddiqui AH, Rudin S, Ionita CN. 3D printed abdominal aortic aneurysm phantom for image guided surgical planning with a patient specific fenestrated endovascular graft system. Medical imaging 2017: Imaging informatics for healthcare, research, and applications. SPIE. 2017;10138:159–172. DOI: 10.1117/12.2253902.

[2] Kodenko MR, Vasilev YuA, Vladzymyrskyy AV, Omelyanskaya OV, Leonov DV, Blokhin IA, Novik VP, Kulberg NS, Samorodov AV, Mokienko OA, Reshetnikov RV. Diagnostic accuracy of AI for opportunistic screening of abdominal aortic aneurysm in CT: A systematic review and narrative synthesis. Diagnostics. 2022;12(12):3197. DOI: 10.3390/diagnostics12123197.

[3] Le Berre C, Sandborn WJ, Aridhi S, Devignes M-D, Fournier L, Smaïl-Tabbone M, Danese S, Peyrin-Biroulet L. Application of artificial intelligence to gastroenterology and hepatology. Gastroenterology. 2020;158(1):76–94.e2. DOI: 10.1053/j.gastro.2019.08.058.

[4] Medar R, Rajpurohit VS, Rashmi B. Impact of training and testing data splits on accuracy of time series forecasting in machine learning. 2017 International Conference on Computing, Communication, Control and Automation (ICCUBEA). Pune, India: IEEE; 2017. с. 1–6. DOI: 10.1109/ICCUBEA.2017.8463779.

[5] Pavlov NA, Andreychenko AE, Vladzymyrskyy AV, Revazyan AA, Kirpichev YuS, Morozov SP. Reference medical datasets (MosMedData) for independent external evaluation of algorithms based on artificial intelligence in diagnostics. Digital Diagnostics. 2021;2(1):49–65. (In Russ.) DOI: 10.17816/DD60635.

[6] Павлов Н.А., Андрейченко А.Е., Владзимирский А.В., Ревазян А.А., Кирпичев Ю.С., Морозов С.П. Эталонные медицинские датасеты (MosMedData) для независимой внешней оценки алгоритмов на основе искусственного интеллекта в диагностике. Digital Diagnostics. 2021;2(1):49–65. DOI: 10.17816/DD60635.

[7] Wallner J, Schwaiger M, Hochegger K, Gsaxner C, Zemann W, Egger J. A review on multiplatform evaluations of semi-automatic open-source based image segmentation for cranio-maxillofacial surgery. Comput Methods Programs Biomed. 2019;182:105102. DOI: 10.1016/j.cmpb.2019.105102.

[8] Free software for annotating DICOM in deep learning. https://www.imaios.com/en/resources/blog/the-best-medical-image-annotation-software (access date: 20.01.2023).

[9] Grünber K, Jimenez-del-Toro O, Jakab A, Langs G, Salas Fernandez T, Winterstein M, Krenn M. Annotating medical image data. In: Cloud-Based Benchmarking of Medical Image Analysis. Cham: Springer; 2017. p. 45–67. DOI: 10.1007/978-3-319-49644-3.

[10] 3D Slicer image computing platform. In: 3D Slicer image computing platform. https://www.slicer.org/ (access date: 15.01.2023).

[11] ITK-SNAP. http://www.itksnap.org/pmwiki/pmwiki.php (access date: 15.01.2023).

[12] Unified OS/Platform for computer vision. https://supervise.ly/ (access date: 16.01.2023).

[13] The Medical Imaging Interaction Toolkit. https://github.com/MITK/MITK?ysclid=lfrblsfeel784158071 (access date: 16.01.2023).

[14] MedSeg — free medical segmentation online. https://www.medseg.ai/ (access date: 16.01.2023).

[15] Computer vision annotation tool. https://www.cvat.ai/ (access date: 16.01.2023).

[16] Vaa3D. https://en.wikipedia.org/wiki/Vaa3D (access date: 10.01.2023).

[17] СellProfiler. Cell image analysis software. https://cellprofiler.org/ (access date: 10.01.2023).

[18] Krasil'nikov NN. Tsifrovaya obrabotka 2D- i 3D-izobrazheniy. (Digital processing of 2D and 3D images.) SPb.: BHV-Peterburg; 2011. 608 р. (In Russ.)

[19] Красильников Н.Н. Цифровая обработка 2D- и 3D-изображений. СПб.: БХВ-Петербург; 2011. 608 с.

[20] Extensions. https://slicer.readthedocs.io/en/latest/developer_guide/extensions.html (access date: 16.01.2023).

[21] Creating a Supervisely plugin. https://sdk.docs.supervise.ly/repo/help/tutorials/01_create_new_plugin/how_to_create_plugin.html (access date: 02.02.2023).

[22] Serverless tutorial. https://opencv.github.io/cvat/docs/manual/advanced/serverless-tutorial/ (access date: 16.01.2023).

[23] Developer Guide. https://slicer.readthedocs.io/en/latest/developer_guide/index.html# (access date: 12.05.2023).

[24] API & SDK. https://opencv.github.io/cvat/docs/api_sdk/ (access date: 12.05.2023).

[25] API & SDK Seamless integration with existing codebase. https://supervisely.com/ecosystem/api-and-sdk/ (access date: 12.05.2023).

[26] DSS Service Developer’s Quick Start Guide. https://alfabis-server.readthedocs.io/en/latest/service_quick_start.html (access date: 16.01.2023).

[27] Rogowska J. Overview and fundamentals of medical image segmentation. In: Handbook of medical imaging, processing and analysis. Academic Press, Cambridge, Massachusetts; 2000. с. 69–85. DOI: 10.1016/B978-012077790-7/50009-6.

[28] Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage. 2006;31(3):1116–1128. DOI: 10.1016/j.neuroimage.2006.01.015.

[29] Fast Marching Segmentation. https://simpleitk.readthedocs.io/en/master/link_FastMarchingSegmentation_docs.html (access date: 17.04.2023).

[30] Mortensen EN, Barrett WA. Intelligent scissors for image composition. In: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. New York, NY: Association for Computing Machinery; 1995. р. 191–198. DOI: 10.1145/218380.218442.

[31] Zuki D, Vicory J, McCormick M, Wisse LEM, Gerig G, Yushkevich P, Aylward S. ND morphological contour interpolation. Insight J. 2016;17:1–27. DOI: 10.54294/achtrg.

[32] SlicerSegmentEditorExtraEffects. https://github.com/lassoan/SlicerSegmentEditorExtraEffects (access date: 17.04.2023)

[33] Zhu L, Kolesov I, Gao Y, Kikinis R, Tannenbaum A. An effective interactive medical image segmentation method using fast GrowCut. http://interactivemedical.org/imic2014/CameraReadyPapers/Paper%204/IMIC_ID4_FastGrowCut.pdf (access date: 16.01.2023).

[34] Breiman L. Random Forests. Journal Machine Learning. 2001;45(1):5–32. DOI: 10.1023/A:1010933404324.

[35] Wang J, Sun K, Cheng T, Jiang B, Deng C, Zhao Y. Deep High-Resolution Representation Learning for Visual Recognition. IEEE Trans Pattern Anal Mach Intell. 2021;43(10):3349–3364. DOI: 10.1109/TPAMI.2020.2983686.

[36] Maninis KK, Caelles S, Pont-Tuset J, Van Gool L. Deep extreme cut: From extreme points to object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, UT, USA; 2018. р. 616–625, DOI: 10.48550/arXiv.1711.09081.

[37] Sofiiuk K, Petrov I, Barinova O, Konushin A. Rethinking backpropagating refinement for interactive segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Seattle, WA, USA; 2020. р. 8623–8632, DOI: 10.48550/arXiv.2001.10331.

[38] Sakinis T, Milletari F, Roth H, Korfiatis P, Kostandy P, Philbrick K, Erickson BJ. Interactive segmentation of medical images through fully convolutional neural networks. ArXiv preprint. 2019. DOI: 10.48550/arXiv.1903.08205.

[39] Sofiiuk K, Petrov IA, Konushin A. Reviving Iterative Training with Mask Guidance for Interactive Segmentation, 2022. In: IEEE International Conference on Image Processing (ICIP). Bordeaux; 2022. р. 3141–3145. DOI: 10.1109/ICIP46576.2022.9897365.

[40] ABL Temporal Bone Segmentation Module. https://github.com/Auditory-Biophysics-Lab/Slicer-ABLTemporalBoneSegmentation (access date: 17.04.2023).

[41] Airway Segmentation. GitHub — Slicer/SlicerAirwaySegmentation: CLI module for airway segmentation starting from chest CT images (access date: 17.04.2023).

[42] Breast DCE-MRI FTV extension. https://github.com/rnadkarni2/SlicerBreast_DCEMRI_FTV (access date: 17.04.2023).

[43] RVesselX Slicer Liver Anatomy Annotation Plugin. https://github.com/R-Vessel-X/SlicerRVXLiverSegmentation (access date: 17.04.2023).

[44] The VMTK Extension for 3D Slicer. https://github.com/vmtk/SlicerExtension-VMTK (access date: 17.04.2023).

[45] HDBrainExtraction. https://github.com/lassoan/SlicerHDBrainExtraction (access date: 17.04.2023).

[46] T1 & ECV Mapping. https://github.com/RivettiLuciano/SlicerT1_ECVMapping (access date: 17.04.2023).

Funding

Автономная некоммерческая организация «Московский центр инновационных технологий в здравоохранении»(ЕГИСУ: 122112400040-1)

RIGHTS & PERMISSIONS

Eco-Vector