A Semantic Segmentation Network for Colorectal Polyp Images With Progressive Fusion of Dual-Branch Features

Abstract

Accurate segmentation of colorectal polyps is essential for early colorectal cancer screening, yet remains challenging due to weak foreground–background contrast, disrupted boundaries caused by specular reflections and intestinal folds, and pronounced scale variation among polyps. These factors make it difficult for existing methods to jointly preserve fine boundary details and robust global semantic context. To address these task-specific challenges, we propose a Dual-branch Feature Progressive Fusion Network (DFPF-Net) for colorectal polyp segmentation. DFPF-Net adopts a dual-encoder architecture that integrates a CNN-based encoder for local and boundary-sensitive representation for global semantic modelling. A boundary-aware branch equipped with stacked Inversely Perceive Information Layers (IPILs) enhances ambiguous and fragmented contours, while the semantic branch incorporates Misalignment Fusion Modules (MFMs) and a Misaligned Single-layer Reinforcement Module (MSRM) to alleviate semantic misalignment and insufficient cross-scale interaction. Furthermore, a Perceptual Information Fusion Module (PIFM) enables effective semantic–boundary collaboration, and a Multi-level Residual Decoding Module (MRDM) progressively reconstructs structurally consistent segmentation outputs. Extensive experiments on multiple public colonoscopy datasets demonstrate that DFPF-Net achieves competitive and robust segmentation performance. In particular, on the challenging ETIS dataset, DFPF-Net attains 0.785 mDice and 0.704 mIoU, indicating its capability in handling complex structures and ambiguous boundaries in colorectal polyp segmentation.

Keywords

boundary-aware learning / colorectal polyp segmentation / multi-scale fusion / semantic segmentation / vision transformer

Acknowledgements

The authors would like to thank the Department of Artificial Intelligence at China Jiliang University for providing computational resources and technical support. We also extend our appreciation to the clinical collaborators at Sir Run Run Shaw Hospital for their valuable assistance in interpreting colonoscopic images.

Funding

This work was supported by the National Natural Science Foundation of China (No. U23A20487), Key R&D Projects in Zhejiang Province (No. 2024C01108), Key R&D Projects in Ningbo (No. 2024Z114), Key R&D Projects in Hangzhou (No. 2024SZD1A09) and 2023 Fundamental Research Funds for the Central Universities of China Jiliang University (No. 2023YW70).

Ethics Statement

The authors have nothing to report.

Consent

This study uses only publicly available de-identified datasets. No individual patient information is involved, and no informed consent is required.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The datasets used in this study are all publicly available and can be accessed through the following sources: Kvasir. https://datasets.simula.no/kvasir-seg/CVC-ClinicDB,CVC-ColonDB, CVC-300,ETlS: https://github.com/DengPingFan/PraNet. PolypGen: https://github.com/sharibox/PolypGen-Benchmark.git. Additional experimental results and code will be made available upon reasonable request to the corresponding author.

Permission to Reproduce Material From Other Sources

All datasets and materials used in this study are publicly available and properly cited. No third-party materials requiring copyright permission were used.

References

[1] Z. Wu, F. Lv, C. Chen, et al., Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey,” arXiv preprint arXiv:2401.11734 (2024).

[2] S. Li, Y. Ren, Y. Yu, Q. Jiang, X. He, and H. Li, “A Survey of Deep Learning Algorithms for Colorectal Polyp Segmentation,” Neurocomputing 614 (2025): 128767, https://doi.org/10.1016/j.neucom.2024.128767.

[3] A. Qayoom, J. Xie, and H. Ali, “Polyp Segmentation in Medical Imaging: Challenges, Approaches and Future Directions,” Artificial Intelligence Review 58, no. 6 (2025): 169, https://doi.org/10.1007/s10462-025-11173-2.

[4] J. Silva, A. Histace, O. Romain, X. Dray, and B. Granado, “Toward Embedded Detection of Polyps in Wce Images for Early Diagnosis of Colorectal Cancer,” International Journal of Computer Assisted Radiology and Surgery 9, no. 2 (2014): 283-293, https://doi.org/10.1007/s11548-013-0926-3.

[5] J. Bernal, J. Sánchez, and F. Vilarino, “Towards Automatic Polyp Detection With a Polyp Appearance Model,” Pattern Recognition 45, no. 9 (2012): 3166-3182, https://doi.org/10.1016/j.patcog.2012.03.002.

[6] D. Rajasekar, G. Theja, M. R. Prusty, and S. Chinara, “Efficient Colorectal Polyp Segmentation Using Wavelet Transformation and AdaptUNet: A Hybrid U-Net,” Heliyon 10, no. 13 (2024): e33655, https://doi.org/10.1016/j.heliyon.2024.e33655.

[7] D. P. Fan, G. P. Ji, T. Zhou, et al., Pranet: Parallel Reverse Attention Network for Polyp Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer International Publishing, 2020), 263-273.

[8] X. Zhao, L. Zhang, and H. Lu, “Automatic Polyp Segmentation via Multi-Scale Subtraction Network,” in International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer International Publishing, 2021), 120-130.

[9] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional Networks for Biomedical Image Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer international publishing, 2015), 234-241.

[10] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, et al., Unet++: A Nested u-Net Architecture for Medical Image Segmentation,” in International Workshop on Deep Learning in Medical Image Analysis (Springer International Publishing, 2018), 3-11.

[11] B. Murugesan, K. Sarveswaran, S. M. Shankaranarayana, et al., Psi-Net: Shape and Boundary Aware Joint multi-task Deep Network for Medical Image Segmentation,” in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2019), 7223-7226.

[12] J. Liu, Q. Chen, Y. Zhang, Z. Wang, X. Deng, and J. Wang, “Multi-Level Feature Fusion Network Combining Attention Mechanisms for Polyp Segmentation,” Information Fusion 104 (2024): 102195, https://doi.org/10.1016/j.inffus.2023.102195.

[13] Y. Su, J. Cheng, C. Zhong, C. Jiang, J. Ye, and J. He, “Accurate Polyp Segmentation Through Enhancing Feature Fusion and Boosting Boundary Performance,” Neurocomputing 545 (2023): 126233, https://doi.org/10.1016/j.neucom.2023.126233.

[14] W. Wang, E. Xie, X. Li, et al., Pvt v2: Improved Baselines With Pyramid Vision Transformer,” Computational visual media 8, no. 3 (2022): 415-424, https://doi.org/10.1007/s41095-022-0274-8.

[15] R. Zhang, G. Li, Z. Li, et al., Adaptive Context Selection for Polyp Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer International Publishing, 2020), 253-262.

[16] T. Kim, H. Lee, and D. Kim, “Uacanet: Uncertainty Augmented Context Attention for Polyp Segmentation,” in Proceedings of the 29th ACM International Conference on Multimedia, (2021), 2167-2175.

[17] Y. Fang, C. Chen, Y. Yuan, et al., Selective Feature Aggregation Network With Area-Boundary Constraints for Polyp Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer International Publishing, 2019), 302-310.

[18] T. Zhou, Y. Zhou, K. He, et al., Cross-Level Feature Aggregation Network for Polyp Segmentation,” Pattern Recognition 140 (2023): 109555, https://doi.org/10.1016/j.patcog.2023.109555.

[19] D. Jha, M. A. Riegler, D. Johansen, et al., Doubleu-Net: A Deep Convolutional Neural Network for Medical Image Segmentation,” in 2020 IEEE 33rd International Symposium on Computer-based Medical Systems (CBMS) (IEEE, 2020), 558-564.

[20] C. H. Huang, H. Y. Wu, and Y. L. Lin, “Hardnet-Mseg: A Simple Encoder-Decoder Polyp Segmentation Neural Network That Achieves over 0.9 Mean Dice and 86 Fps,” arXiv preprint arXiv:2101.07172 (2021).

[21] J. Chen, Y. Lu, Q. Yu, et al., Transunet: Transformers Make Strong Encoders for Medical Image Segmentation,” arXiv preprint arXiv:2102.04306 (2021).

[22] Y. Zhang, H. Liu, and Q. Hu, “Transfuse: Fusing Transformers and Cnns for Medical Image Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer International Publishing, 2021), 14-24.

[23] B. Dong, W. Wang, D. P. Fan, et al., Polyp-Pvt: Polyp Segmentation With Pyramid Vision Transformers,” arXiv preprint arXiv:2108.06932 (2021).

[24] N. T. Duc, N. T. Oanh, N. T. Thuy, T. M. Triet, and V. S. Dinh, “Colonformer: An Efficient Transformer Based Method for Colon Polyp Segmentation,” IEEE Access 10 (2022): 80575-80586, https://doi.org/10.1109/access.2022.3195241.

[25] M. Niemeyer, J. T. Barron, B. Mildenhall, et al., Regnerf: Regularizing Neural Radiance Fields for View Synthesis From Sparse Inputs,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2022), 5480-5490.

[26] H. Liu, S. Ma, D. Xia, and S. Li, “SFANet: A Spectrum-Aware Feature Augmentation Network for Visible-Infrared Person Reidentification,” IEEE Transactions on Neural Networks and Learning Systems 34, no. 4 (2021): 1958-1971, https://doi.org/10.1109/tnnls.2021.3105702.

[27] S. Woo, J. Park, J. Y. Lee, et al., Cbam: Convolutional Block Attention Module,” in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 3-19.

[28] Q. Liu, Y. Lin, X. Han, K. Chen, W. Zhang, and H. Yang, “Dual-Branch Feature Extraction Network Combined With Transformer and CNN for Polyp Segmentation,” International Journal of Imaging Systems and Technology 34, no. 1 (2024): e22987, https://doi.org/10.1002/ima.22987.

[29] X. Li, Q. Liu, X. Li, et al., Ciftc-Net: Cross Information Fusion Network With Transformer and Cnn for Polyp Segmentation,” Displays 85 (2024): 102872, https://doi.org/10.1016/j.displa.2024.102872.

[30] Y. Lin, X. Han, K. Chen, W. Zhang, and Q. Liu, “CSwinDoubleU-Net: A Double U-Shaped Network Combined With Convolution and Swin Transformer for Colorectal Polyp Segmentation,” Biomedical Signal Processing and Control 89 (2024): 105749, https://doi.org/10.1016/j.bspc.2023.105749.

[31] W. Liu, Z. Li, C. Li, and H. Gao, “Ectransnet: An Automatic Polyp Segmentation Network Based on Multi-Scale Edge Complementary,” Journal of Digital Imaging 36, no. 6 (2023): 2427-2440, https://doi.org/10.1007/s10278-023-00885-y.

[32] C. Yang and Z. Zhang, “Pfd-Net: Pyramid Fourier Deformable Network for Medical Image Segmentation,” Computers in Biology and Medicine 172 (2024): 108302, https://doi.org/10.1016/j.compbiomed.2024.108302.

[33] T. Chen, X. Zhou, Z. Tan, et al., Zig-Rir: Zigzag rwkv-in-rwkv for Efficient Medical Image Segmentation,” IEEE Transactions on Medical Imaging (2025).

[34] C. Luo, Y. Wang, Z. Deng, et al., Colonic Polyp Segmentation Based on Transformer-Convolutional Neural Networks Fusion,” Pattern Recognition 170 (2025): 112116, https://doi.org/10.1016/j.patcog.2025.112116.

[35] J. Wei, Y. Hu, R. Zhang, et al., Shallow Attention Network for Polyp Segmentation,” in International Conference on Medical Image Computing and computer-assisted Intervention (Springer International Publishing, 2021), 699-708.

[36] L. Lin, G. Lv, B. Wang, C. Xu, and J. Liu, “Polyp-LVT: Polyp Segmentation With Lightweight Vision Transformers,” Knowledge-Based Systems 300 (2024): 112181, https://doi.org/10.1016/j.knosys.2024.112181.

[37] M. A. Manan, F. Jinchao, S. Ahmed, and A. Raheem, “Enhancing Colorectal Polyp Segmentation With TCFMA-Net: A Transformer-Based Cross Feature and Multi-Attention Network,” Artificial Intelligence in Medicine 167 (2025): 103167, https://doi.org/10.1016/j.artmed.2025.103167.

[38] M. A. Manan, J. Feng, S. Ahmed, and A. Raheem, “MSCR-Net: Multipath Separable Convolution-Based Residual Network for Segmentation of Colorectal Polyp,” Biomedical Signal Processing and Control 112 (2026): 108674, https://doi.org/10.1016/j.bspc.2025.108674.

[39] J. Bernal, N. Tajkbaksh, F. J. Sanchez, et al., Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results From the MICCAI 2015 Endoscopic Vision Challenge,” IEEE Transactions on Medical Imaging 36, no. 6 (2017): 1231-1249, https://doi.org/10.1109/tmi.2017.2664042.

[40] A. Q. Khan, G. Sun, A. Bilal, et al., Optimizing Fully Convolutional Encoder-Decoder Network for Segmentation of Diabetic Eye Disease,” Computers, Materials & Continua 77, no. 2 (2023): 2481-2504, https://doi.org/10.32604/cmc.2023.043239.

[41] H. Guan and M. Liu, “Domain Adaptation for Medical Image Analysis: A Survey,” IEEE Transactions on Biomedical Engineering 69, no. 3 (2021): 1173-1185, https://doi.org/10.1109/tbme.2021.3117407.

[42] M. Yaqub, S. Ahmad, M. A. Manan, M. S. Pathan, and L. He, “Predicting Traffic Flow With Federated Learning and Graph Neural With Asynchronous Computations Network,” Array 26 (2025): 100411, https://doi.org/10.1016/j.array.2025.100411.