Artificial intelligence in capsule endoscopy: development status and future expectations

Abstract

In this review, we aim to illustrate the state-of-the-art artificial intelligence (AI) applications in the field of capsule endoscopy. AI has made significant strides in gastrointestinal imaging, particularly in capsule endoscopy - a non-invasive procedure for capturing gastrointestinal tract images. However, manual analysis of capsule endoscopy videos is labour-intensive and error-prone, prompting the development of automated computational algorithms and AI models. While currently serving as a supplementary observer, AI has the capacity to evolve into an autonomous, integrated reading system, potentially significantly reducing capsule reading time while surpassing human accuracy. We searched Embase, Pubmed, Medline, and Cochrane databases from inception to 06 Jul 2023 for studies investigating the use of AI for capsule endoscopy and screened retrieved records for eligibility. Quantitative and qualitative data were extracted and synthesised to identify current themes. In the search, 824 articles were collected, and 291 duplicates and 31 abstracts were deleted. After a double-screening process and full-text review, 106 publications were included in the review. Themes pertaining to AI for capsule endoscopy included active gastrointestinal bleeding, erosions and ulcers, vascular lesions and angiodysplasias, polyps and tumours, inflammatory bowel disease, coeliac disease, hookworms, bowel prep assessment, and multiple lesion detection. This review provides current insights into the impact of AI on capsule endoscopy as of 2023. AI holds the potential for faster and precise readings and the prospect of autonomous image analysis. However, careful consideration of diagnostic requirements and potential challenges is crucial. The untapped potential within vision transformer technology hints at further evolution and even greater patient benefit.

Keywords

Artificial intelligence / capsule endoscopy / computer-assisted diagnosis / computer-assisted detection / deep learning / vision transformer / review

References

[1] Muñoz-Navas M.Capsule endoscopy.World J Gastroenterol2009;15:1584-6 PMCID:PMC2669941

[2] Beg S, Card T, Sidhu R, Wronska E, Ragunath K; UK capsule endoscopy users’ group. The impact of reader fatigue on the accuracy of capsule endoscopy interpretation. Dig Liver Dis 2021;53:1028-33.

[3] Giritharan B,Liu J,Oh JH.Bleeding detection from capsule endoscopy videos.Annu Int Conf IEEE Eng Med Biol Soc2008;2008:4780-3

[4] Pan G,Song X.BP neural network classification for bleeding detection in wireless capsule endoscopy.J Med Eng Technol2009;33:575-81

[5] Ghosh T,Shahnaz C.An automatic bleeding detection scheme in wireless capsule endoscopy based on histogram of an RGB-indexed image.Annu Int Conf IEEE Eng Med Biol Soc2014;2014:4683-6

[6] Hassan AR.Computer-aided gastrointestinal hemorrhage detection in wireless capsule endoscopy videos.Comput Methods Programs Biomed2015;122:341-53

[7] Pan G,Qiu X.Bleeding detection in wireless capsule endoscopy based on probabilistic neural network.J Med Syst2011;35:1477-84

[8] Li B.Computer-aided detection of bleeding regions for capsule endoscopy images.IEEE Trans Biomed Eng2009;56:1032-9

[9] Yuan Y,Meng MQH.Bleeding frame and region detection in the wireless capsule endoscopy video.IEEE J Biomed Health Inform2016;20:624-30

[10] Ghosh T,Wahid KA,Ahmad MO.Cluster based statistical feature extraction method for automatic bleeding detection in wireless capsule endoscopy video.Comput Biol Med2018;94:41-54

[11] Pogorelov K,Azmadi Hussin F.Bleeding detection in wireless capsule endoscopy videos - Color versus texture features.J Appl Clin Med Phys2019;20:141-54 PMCID:PMC6698770

[12] Rathnamala S.Automated bleeding detection in wireless capsule endoscopy images based on color feature extraction from Gaussian mixture model superpixels.Med Biol Eng Comput2021;59:969-87

[13] Jia X.A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images.Annu Int Conf IEEE Eng Med Biol Soc2016;2016:639-42

[14] Jia X.Gastrointestinal bleeding detection in wireless capsule endoscopy images using handcrafted and CNN features.Annu Int Conf IEEE Eng Med Biol Soc2017;2017:3154-7

[15] Hajabdollahi M,Najarian K,Samavi S.Low complexity CNN structure for automatic bleeding zone detection in wireless capsule endoscopy imaging.Annu Int Conf IEEE Eng Med Biol Soc2019;2019:7227-30

[16] Kanakatte A.Precise bleeding and red lesions localization from capsule endoscopy using compact U-net.Annu Int Conf IEEE Eng Med Biol Soc2021;2021:3089-92

[17] Ghosh T.Deep transfer learning for automated intestinal bleeding detection in capsule endoscopy imaging.J Digit Imaging2021;34:404-17 PMCID:PMC8290011

[18] Ribeiro T,Ferreira JPS.Artificial intelligence and capsule endoscopy: automatic detection of vascular lesions using a convolutional neural network.Ann Gastroenterol2021;34:820-8 PMCID:PMC8596215

[19] Mascarenhas Saraiva M,Afonso J.Artificial intelligence and capsule endoscopy: automatic detection of small bowel blood content using a convolutional neural network.GE Port J Gastroenterol2022;29:331-8 PMCID:PMC9485980

[20] Muruganantham P.Attention aware deep learning model for wireless capsule endoscopy lesion classification and localization.J Med Biol Eng2022;42:157-68

[21] Kundu AK,Rizve MN.An automatic bleeding frame and region detection scheme for wireless capsule endoscopy videos based on interplane intensity variation profile in normalized RGB color space.J Healthc Eng2018;2018:9423062 PMCID:PMC5845490

[22] Xing X,Meng MQH.Bleeding detection in wireless capsule endoscopy image video using superpixel-color histogram and a subspace KNN classifier.Annu Int Conf IEEE Eng Med Biol Soc2018;2018:1-4

[23] Charisis V,Liatsos CN,Sergiadis GD.Abnormal pattern detection in wireless capsule endoscopy images using nonlinear analysis in RGB color space. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology; 2010 Aug 31 - Sep 04; Buenos Aires, Argentina. IEEE; 2010. pp. 3674-7.

[24] Li B.Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments.Comput Biol Med2009;39:141-7

[25] Charisis VS,Liatsos CN,Sergiadis GD.Capsule endoscopy image analysis using texture information from various colour models.Comput Methods Programs Biomed2012;107:61-74

[26] Iakovidis DK.Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software.Gastrointest Endosc2014;80:877-83

[27] Fan S,Fan Y,Li L.Computer-aided detection of small intestinal ulcer and erosion in wireless capsule endoscopy images.Phys Med Biol2018;63:165001

[28] Khan MA,Akram T,Nayak RS.Stomach deformities recognition using rank-based deep features selection.J Med Syst2019;43:329

[29] Kundu AK,Wahid KA.Multiple linear discriminant models for extracting salient characteristic patterns in capsule endoscopy images for multi-disease detection.IEEE J Transl Eng Health Med2020;8:3300111 PMCID:PMC7062148

[30] Wang S,Zhang L,Zhang H.A systematic evaluation and optimization of automatic detection of ulcers in wireless capsule endoscopy on a large dataset using deep convolutional neural networks.Phys Med Biol2019;64:235014

[31] Aoki T,Aoyama K.Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc2019;89:357-63.e2

[32] Ding Z,Zhang H.Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model.Gastroenterology2019;157:1044-54.e5

[33] Majid A,Yasmin M,Yousafzai A.Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection.Microsc Res Tech2020;83:562-76

[34] Otani K,Kurose Y.Automatic detection of different types of small-bowel lesions on capsule endoscopy images using a newly developed deep convolutional neural network.Endoscopy2020;52:786-91

[35] Xia J,Pan J.Use of artificial intelligence for detection of gastric lesions by magnetically controlled capsule endoscopy.Gastrointest Endosc2021;93:133-9.e4

[36] Afonso J,Ferreira JPS.Development of a convolutional neural network for detection of erosions and ulcers with distinct bleeding potential in capsule endoscopy.Tech Innov Gastrointest Endosc2021;23:291-6

[37] Mascarenhas Saraiva MJ,Ribeiro T.Deep learning and capsule endoscopy: automatic identification and differentiation of small bowel lesions with distinct haemorrhagic potential using a convolutional neural network.BMJ Open Gastroenterol2021;8:e000753 PMCID:PMC8477239

[38] Afonso J,Ferreira JPS.Automated detection of ulcers and erosions in capsule endoscopy images using a convolutional neural network.Med Biol Eng Comput2022;60:719-25

[39] Mascarenhas M,Afonso J.Deep learning and colon capsule endoscopy: automatic detection of blood and colonic mucosal lesions using a convolutional neural network.Endosc Int Open2022;10:E171-7 PMCID:PMC8850002

[40] Xiao P,Cai F.A deep learning based framework for the classification of multi- class capsule gastroscope image in gastroenterologic diagnosis.Front Physiol2022;13:1060591 PMCID:PMC9716070

[41] Ribeiro T,Afonso J.Artificial intelligence and colon capsule endoscopy: automatic detection of ulcers and erosions using a convolutional neural network.J Gastroenterol Hepatol2022;37:2282-8

[42] Raut V,Shete VV.Gastrointestinal tract disease segmentation and classification in wireless capsule endoscopy using intelligent deep learning model.Comput Method Biomech Biomed Eng Imaging Vis2023;11:606-22

[43] Nakada A,Otani K.Improved object detection artificial intelligence using the revised RetinaNet model for the automatic detection of ulcerations, vascular lesions, and tumors in wireless capsule endoscopy.Biomedicines2023;11:942 PMCID:PMC10046454

[44] Gan T,Rao NN,Liu B.A feasibility trial of computer-aided diagnosis for enteric lesions in capsule endoscopy.World J Gastroenterol2008;14:6929-35 PMCID:PMC2773855

[45] Arieira C,de Castro FD.Capsule endoscopy: is the software TOP 100 a reliable tool in suspected small bowel bleeding?.Dig Liver Dis2019;51:1661-4

[46] Vieira PM,Costa D,Rolanda C.Automatic segmentation and detection of small bowel angioectasias in WCE images.Ann Biomed Eng2019;47:1446-62

[47] Vezakis IA,Polydorou AA.A novel real-time automatic angioectasia detection method in wireless capsule endoscopy video feed.Annu Int Conf IEEE Eng Med Biol Soc2019;2019:4072-5

[48] Leenhardt R, Vasseur P, Li C, et al; The CAD-CAP Database Working Group. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2019;89:189-94.

[49] Tsuboi A,Aoyama K.Artificial intelligence using a convolutional neural network for automatic detection of small-bowel angioectasia in capsule endoscopy images.Dig Endosc2020;32:382-90

[50] Aoki T,Kato Y.Automatic detection of various abnormalities in capsule endoscopy videos by a deep learning-based system: a multicenter study.Gastrointest Endosc2021;93:165-73.e1

[51] Hwang Y,Park C.Improved classification and localization approach to small bowel capsule endoscopy using convolutional neural network.Dig Endosc2021;33:598-607

[52] Hosoe N,Tojo A.Development of a deep-learning algorithm for small bowel-lesion detection and a study of the improvement in the false-positive rate.J Clin Med2022;11:3682 PMCID:PMC9267395

[53] Chu Y,Gao M.Convolutional neural network-based segmentation network applied to image recognition of angiodysplasias lesion under capsule endoscopy.World J Gastroenterol2023;29:879-89 PMCID:PMC9932427

[54] Leenhardt R,Li Cynthia.403 A highly sensitive and highly specific convolutional neural network-based algorithm for automated diagnosis of angiodysplasia in small bowel capsule endoscopy.Gastrointest Endosc2018;87:AB78

[55] Li B,Lau JYW.Computer-aided small bowel tumor detection for capsule endoscopy.Artif Intell Med2011;52:11-6

[56] Karargyris A.Detection of small bowel polyps and ulcers in wireless capsule endoscopy videos.IEEE Trans Biomed Eng2011;58:2777-86

[57] Barbosa DC,Ramos JC,Lima CS.Automatic small bowel tumor diagnosis by using multi-scale wavelet-based analysis in wireless capsule endoscopy images.Biomed Eng Online2012;11:3 PMCID:PMC3296640

[58] Mamonov AV,Figueiredo PN.Automated polyp detection in colon capsule endoscopy.IEEE Trans Med Imaging2014;33:1488-502

[59] Liu G,Kuang S.Detection of small bowel tumor based on multi-scale curvelet analysis and fractal technology in capsule endoscopy.Comput Biol Med2016;70:131-8

[60] Yang J,Li S,Zhu T.WCE polyp detection based on novel feature descriptor with normalized variance locality-constrained linear coding.Int J Comput Assist Radiol Surg2020;15:1291-302

[61] Vieira PM,Valente J,Rolanda C.Automatic detection of small bowel tumors in wireless capsule endoscopy images using ensemble learning.Med Phys2020;47:52-63

[62] Yuan Y.Deep learning for polyp recognition in wireless capsule endoscopy images.Med Phys2017;44:1379-89

[63] Saito H,Aoyama K.Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc2020;92:144-51.e1

[64] Yamada A,Otani K,Koike K.Automatic detection of colorectal neoplasia in wireless colon capsule endoscopic images using a deep convolutional neural network.Endoscopy2021;53:832-6

[65] Saraiva MM,Cardoso H.Artificial intelligence and colon capsule endoscopy: development of an automated diagnostic system of protruding lesions in colon capsule endoscopy.Tech Coloproctol2021;25:1243-8

[66] Jain S,Ojha A.A deep CNN model for anomaly detection and localization in wireless capsule endoscopy images.Comput Biol Med2021;137:104789

[67] Zhou JX,Xi DH.Enhanced segmentation of gastrointestinal polyps from capsule endoscopy images with artifacts using ensemble learning.World J Gastroenterol2022;28:5931-43 PMCID:PMC9669827

[68] Mascarenhas M,Ribeiro T.Performance of a deep learning system for automatic diagnosis of protruding lesions in colon capsule endoscopy.Diagnostics2022;12:1445 PMCID:PMC9222144

[69] Gilabert P,Laiz P.Artificial intelligence to improve polyp detection and screening time in colon capsule endoscopy.Front Med2022;9:1000726 PMCID:PMC9606587

[70] Liu F,Li J.DBMF: dual branch multiscale feature fusion network for polyp segmentation.Comput Biol Med2022;151:106304

[71] Souaidi M,Kerkaou Z,Koutti L.A multiscale polyp detection approach for GI tract images based on improved DenseNet and single-shot multibox detector.Diagnostics2023;13:733 PMCID:PMC9955440

[72] Mascarenhas Saraiva M,Ribeiro T.Artificial intelligence and capsule endoscopy: automatic detection of enteric protruding lesions using a convolutional neural network.Rev Esp Enferm Dig2023;115:75-9

[73] Lafraxo S,El Ansari M.Semantic segmentation of digestive abnormalities from WCE images by using AttResU-Net architecture.Life2023;13:719 PMCID:PMC10051085

[74] Blanes-Vidal V,Nadimi ES.Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning.Acta Oncol2019;58:S29-36

[75] Piccirelli S,Bellumat A.New generation express view: an artificial intelligence software effectively reduces capsule endoscopy reading times.Diagnostics2022;12:1783 PMCID:PMC9332221

[76] Eluxeo meets artificial intelligence. Available from: https://asset.fujifilm.com/www/uk/files/2021-05/8fbe51b9718df4e16e3e3a545fa5593a/ELUXEO_CADEYE_Brochure.pdf. [Last accessed on 11 Mar 2024]

[77] Lei II,White E.Study of capsule endoscopy delivery at scale through enhanced artificial intelligence-enabled analysis (the CESCAIL study).Colorectal Dis2023;25:1498-505

[78] Kumar R,Seshamani S,Hager G.Assessment of Crohn’s disease lesions in wireless capsule endoscopy images.IEEE Trans Biomed Eng2012;59:355-62

[79] Haji-Maghsoudi O,Soltanian-Zadeh H.Segmentation of Crohn, lymphangiectasia, xanthoma, lymphoid hyperplasia and stenosis diseases in WCE.Stud Health Technol Inform2012;180:143-7

[80] Charisis VS.Potential of hybrid adaptive filtering in inflammatory lesion detection from capsule endoscopy images.World J Gastroenterol2016;22:8641-57 PMCID:PMC5075542

[81] Barash Y,Soffer S.Ulcer severity grading in video capsule images of patients with Crohn’s disease: an ordinal neural network solution.Gastrointest Endosc2021;93:187-92

[82] de Maissin A,Le Berre C.P161 Computer aided detection of Crohn’s disease small bowel lesions in wireless capsule endoscopy.J Crohns Colitis2018;12:S178-9

[83] Klang E,Margalit R.P285 Deep learning for automated detection of mucosal inflammation by capsule endoscopy in Crohn’s disease.J Crohns Colitis2019;13:S242

[84] Klang E,Margalit RY.Deep learning algorithms for automated detection of Crohn’s disease ulcers by video capsule endoscopy.Gastrointest Endosc2020;91:606-13.e2

[85] de Maissin A,Flamant M.Multi-expert annotation of Crohn’s disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network.Endosc Int Open2021;9:E1136-44 PMCID:PMC8216776

[86] Klang E,Soffer S.Automated detection of Crohn’s disease intestinal strictures on capsule endoscopy images using deep neural networks.J Crohns Colitis2021;15:749-56

[87] Klang E,Mortensen B.A convolutional neural network deep learning model trained on CD ulcers images accurately identifies NSAID ulcers.Front Med2021;8:656493 PMCID:PMC8429810

[88] Majtner T,Herp J,Halling ML.A deep learning framework for autonomous detection and classification of Crohn’s disease lesions in the small bowel and colon with capsule endoscopy.Endosc Int Open2021;9:E1361-70 PMCID:PMC8367448

[89] Ferreira JPS,Afonso JPL.Identification of ulcers and erosions by the novel pillcam™ Crohn’s capsule using a convolutional neural network: a multicentre pilot study.J Crohns Colitis2022;16:169-72

[90] Higuchi N,Sasaki Y.Automated evaluation of colon capsule endoscopic severity of ulcerative colitis using ResNet50.PLoS One2022;17:e0269728 PMCID:PMC9187078

[91] Kratter T,Lev Y.Deep learning multi-domain model provides accurate detection and grading of mucosal ulcers in different capsule endoscopy types.Diagnostics2022;12:2490 PMCID:PMC9600959

[92] Mascarenhas M,Ribeiro T.Deep learning and minimally invasive endoscopy: automatic classification of pleomorphic gastric lesions in capsule endoscopy.Clin Transl Gastroenterol2023;14:e00609 PMCID:PMC10584281

[93] Zhou T,Li BN.Quantitative analysis of patients with celiac disease by video capsule endoscopy: a deep learning method.Comput Biol Med2017;85:1-6

[94] Wang X,Ciaccio EJ.Celiac disease diagnosis from videocapsule endoscopy images with residual learning and deep feature extraction.Comput Methods Programs Biomed2020;187:105236

[95] Li BN,Wang R.Celiac disease detection from videocapsule endoscopy images using strip principal component analysis.IEEE/ACM Trans Comput Biol Bioinform2021;18:1396-404

[96] Chetcuti Zammit S,Greenblatt E.Quantification of celiac disease severity using video capsule endoscopy: a comparison of human experts and machine learning algorithms.Curr Med Imaging2023;19:1455-662 PMCID:PMC10364343

[97] Wu X,Gan T,Ngo CW.Automatic hookworm detection in wireless capsule endoscopy images.IEEE Trans Med Imaging2016;35:1741-52

[98] He JY,Jiang YG,Jain R.Hookworm detection in wireless capsule endoscopy images with deep learning.IEEE Trans Image Process2018;27:2379-92

[99] Gan T,Liu S.Automatic detection of small intestinal hookworms in capsule endoscopy images based on a convolutional neural network.Gastroenterol Res Pract2021;2021:5682288 PMCID:PMC8635910

[100] Nam JH,Oh DJ.Development of a deep learning-based software for calculating cleansing score in small bowel capsule endoscopy.Sci Rep2021;11:4417 PMCID:PMC7904767

[101] Park J,Nam JH.Artificial intelligence that determines the clinical significance of capsule endoscopy images can increase the efficiency of reading.PLoS One2020;15:e0241474 PMCID:PMC7595411

[102] Xing X,Meng MQH.Zoom in lesions for better diagnosis: attention guided deformation network for WCE image classification.IEEE Trans Med Imaging2020;39:4047-59

[103] Zhu M,Yuan Y.DSI-Net: deep synergistic interaction network for joint classification and segmentation with endoscope images.IEEE Trans Med Imaging2021;40:3315-25

[104] Guo X,Hao Y,Liu Z.Multiple abnormality classification in wireless capsule endoscopy images based on EfficientNet using attention mechanism.Rev Sci Instrum2021;92:094102

[105] Goel N,Gunjan D.Investigating the significance of color space for abnormality detection in wireless capsule endoscopy images.Biomed Signal Proces2022;75:103624

[106] Yokote A,Kawasaki K.Small bowel capsule endoscopy examination and open access database with artificial intelligence: the SEE-artificial intelligence project.DEN Open2024;4:e258 PMCID:PMC10288072

[107] Ding Z,Zhang H.Artificial intelligence-based diagnosis of abnormalities in small-bowel capsule endoscopy.Endoscopy2023;55:44-51

[108] LeCun Y,Hinton G.Deep learning.Nature2015;521:436-44

[109] Rudin C.Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nat Mach Intell2019;1:206-15 PMCID:PMC9122117

[110] Koulaouzidis A,Yung DE.KID project: an internet-based digital video atlas of capsule endoscopy for research purposes.Endosc Int Open2017;5:E477-83 PMCID:PMC5452962

[111] Deeba F,Bui FM.Performance assessment of a bleeding detection algorithm for endoscopic video based on classifier fusion method and exhaustive feature selection.Biomed Signal Proces2018;40:415-24

[112] Smedsrud PH,Hicks SA.Kvasir-capsule, a video capsule endoscopy dataset.Sci Data2021;8:142

[113] Bernal J,Fernández-Esparrach G,Rodríguez C.WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians.Comput Med Imaging Graph2015;43:99-111

[114] Coelho P,Leite A,Cunha A.A deep learning approach for red lesions detection in video capsule endoscopies. In: Campilho A, Karray F, ter Haar Romeny B, editors. ICIAR 2018: Image analysis and recognition. Springer, Cham; 2018. pp. 553-61.

[115] Jha D,Riegler MA.Kvasir-SEG: a segmented polyp dataset. In: MMM 2020: MultiMedia modeling. Springer, Cham; 2020. pp. 451-62.

[116] Bernal J,Vilariño F.Towards automatic polyp detection with a polyp appearance model.Pattern Recogn2012;45:3166-82