Endoscopy-assisted lightweight diagnosis system based on transformers for colon polyp detection

Abstract

The integration of endoscopy has significantly propelled the diagnosis and treatment of gastrointestinal diseases, with colonoscopy establishing itself as the primary method for early diagnosis and preventive care in colorectal cancer (CRC). Although deep learning holds promise in mitigating missed polyp rates, modern endoscopy examinations pose additional challenges, such as image blurring and atomizing. This study explores lightweight yet powerful attention mechanisms, introducing the spatial-channel transformer (SCT), an innovative approach that leverages spatial channel relationships for attention weight calculation. The method utilizes rotation operations for inter-dimensional dependencies, followed by residual transformation, encoding inter-channel and spatial information with minimal computational overhead. Extensive experiments on the CVC-ClinicDB polyp detection dataset, addressing endoscopy pitfalls, underscore the superiority of our SCT over other state-of-the-art methods. The proposed model maintains high performance, even in challenging scenarios.

References

[1] PRAKTIKNJO M, SHAWCROSS D, LALEMAN W. The clinical relevance of acute-on-chronic liver failure in medical procedures: endoscopy, interventions and surgery[J]. Liver international, 2023.

[2] Moon R H, Puttock E J, Chen W, et al.. Association of endoscopic biopsy sampling methods with detection of precursor lesions of gastric cancer. Gastrointestinal endoscopy, 2024, 99(2): 204-213 J]

[3] Lima T F N, Bitran J, Frech F S, et al.. Prevalence of post-prostatectomy erectile dysfunction and a review of the recommended therapeutic modalities. International journal of impotence research, 2021, 33(4): 401-409 J]

[4] Akimoto N, Ugai T, Zhong R, et al.. Rising incidence of early-onset colorectal cancer-a call to action. Nature reviews clinical oncology, 2021, 18(4): 230-243 J]

[5] Paing M P, Pintavirooj C. Adenoma dysplasia grading of colorectal polyps using fast Fourier convolutional ResNet (FFC-ResNet). IEEE access, 2023, 11: 16644-16656 J]

[6] Jiang W, Xin L, Zhu S, et al.. Risk factors related to polyp miss rate of short-term repeated colonoscopy. Digestive diseases and sciences, 2023, 68(5): 2040-2049 J]

[7] Yu J, Ma T, Chen H, et al.. Marrying global-local spatial context for image patches in computer-aided assessment. IEEE transactions on systems, man, and cybernetics: systems, 2023, 53(11): 7099-7111 J]

[8] Hassan C, Spadaccini M, Mori Y, et al.. Real-time computer-aided detection of colorectal neoplasia during colonoscopy: a systematic review and meta-analysis. Annals of internal medicine, 2023, 176(9): 1209-1220 J]

[9] Yu J, Ma T, Fu Y, et al.. Local-to-global spatial learning for whole-slide image representation and classification. Computerized medical imaging and graphics, 2023, 107: 102230 J]

[10] Houwen B B S L, Hazewinkel Y, Giotis I, et al.. Computer-aided diagnosis for optical diagnosis of diminutive colorectal polyps including sessile serrated lesions: a real-time comparison with screening endoscopists. Endoscopy, 2023, 55(08): 756-765 J]

[11] Qadir H A, Balasingham I, Solhusvik J, et al.. Improving automatic polyp detection using CNN by exploiting temporal dependency in colonoscopy video. IEEE journal of biomedical and health informatics, 2019, 24(1): 180-193 J]

[12] Shin Y, Qadir H A, Aabakken L, et al.. Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE access, 2018, 6: 40950-40962 J]

[13] Yu J, Gao H, Chen Y, et al.. Deep object detector with attentional spatiotemporal LSTM for space human-robot interaction. IEEE transactions on human-machine systems, 2022, 52(4): 784-793 J]

[14] Wang P, Xiao X, Glissen B J R, et al.. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nature biomedical engineering, 2018, 2(10): 741-748 J]

[15] Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(12): 2481-2495 J]

[16] Urban G, Tripathi P, Alkayali T, et al.. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology, 2018, 155(4): 1069-1078 J]

[17] Girshick R, Donahue J, Darrell T, et al.. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 23–28, 2014, Columbus, OH, USA, 2014 New York IEEE 580-587 [C]

[18] EVERINGHAM M, VAN GOOL L, WILLIAMS C K I, et al. The PASCAL visual object classes challenge 2012 (VOC2012) results[EB/OL]. (2012-05-18) [2023-10-23]. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.

[19] Lin T Y, Maire M, Belongie S, et al.. Microsoft COCO: common objects in context. 13th European Conference on Computer Vision, September 6–12, 2014, Zurich, Switzerland, 2014 Heidelberg Springer International Publishing 740-755 [C]

[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.

[21] Woo S, Park J, Lee J Y, et al.. CBAM: convolutional block attention module. 15th European Conference on Computer Vision, September 8–14, 2018, Munich, Germany, 2018 Heidelberg Springer International Publishing 3-19 [C]

[22] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(8): 7132-7141 J]

[23] Fu J, Liu J, Tian H, et al.. Dual attention network for scene segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 15–20, 2019, Long Bench, CA, USA, 2019 New York IEEE 3146-3154 [C]

[24] Bernal J, Sanchez F J, Fernández-Esparrach G, et al.. WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Computerized medical imaging and graphics, 2015, 43: 99-111 J]

[25] Jha D, Smedsrud P H, Riegler M A, et al.. Kvasir-seg: a segmented polyp dataset. 26th International Conference on MultiMedia Modeling, January 5–8, 2020, Daejeon, South Korea, 2020 Heidelberg Springer International Publishing 451-462 [C]

[26] Yu T, Lin N, Zhang X, et al.. An end-to-end tracking method for polyp detectors in colonoscopy videos. Artificial intelligence in medicine, 2022, 131: 102363 J]

[27] LIMA A C D M, DE PAIVA L F, BRÁZ G, et al. A two-stage method for polyp detection in colonoscopy images based on saliency object extraction and transformers[J]. IEEE access, 2023.

[28] Souaidi M, Lafraxo S, Kerkaou Z, et al.. A multiscale polyp detection approach for GI tract images based on improved dense-net and single-shot multibox detector. Diagnostics, 2023, 13(4): 733 J]

RIGHTS & PERMISSIONS

Tianjin University of Technology