Toward a standardized methodological framework for developing computer vision models in staging laparoscopy

Abstract

Aim: To evaluate deep learning models for anatomical structure and peritoneal metastasis (PM) detection and segmentation during staging laparoscopy (SL) using a phase-independent dataset, and to quantify how annotation strategy and spatial representation relate to predictive performance.

Methods: A checklist covering 25 anatomical structures, one surgical instrument, and PM was defined. Detection models (YOLOv9, Co-DETR) and segmentation models (SegFormer, Mask2Former) were trained under two label configurations. Videos were split at the video level (60/20/20). To quantify annotation distribution and spatial representation, two class-level descriptors were derived from the training set: object count and area fraction (percentage of image area occupied by each class). Class-level associations between these descriptors and test-set performance [F1-score, Intersection over Union (IoU)] were evaluated using Spearman correlation.

Results: Thirty SL videos yielded 2,309 annotated frames (1,304/433/572 for training/validation/testing). YOLOv9 reached mean mAP@50 of 0.52 and 0.61; Mask2Former achieved mean IoU of 0.51 and 0.61 and F1-scores of 0.65 and 0.73 for Sets A and B, respectively. Despite 4,094 annotations, PM remained difficult to segment (IoU 0.29-0.30; F1-score 0.45-0.46), due to low area fraction and high heterogeneity. For IoU, area fraction showed stronger correlations with performance than object count (ρ up to 0.66 vs. 0.48). Similar differences were observed for F1-score.

Conclusions: Anatomical detection and segmentation during SL are feasible but limited by small-target representation and heterogeneous intra-abdominal context. Spatial representation is more closely associated with segmentation performance than annotation frequency, supporting annotation strategies that address sparse pixel coverage in phase-independent intra-abdominal models.

Keywords

Anatomical detection / anatomical segmentation / annotation strategy / computer vision / deep learning / laparoscopic staging / peritoneal metastases

References

[1] Anteby R,Horesh N,Soffer S.et al. Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis Surg Endosc. 2021 35 1521 33

[2] Hashimoto DA,Rosman G,Witkowski ER.et al. Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy Ann Surg. 2019 270 414 21 PMC7216040

[3] Hashimoto DA,Rosman G,Rus D,Meireles OR. Artificial intelligence in surgery: promises and perils Ann Surg. 2018 268 70 6 PMC5995666

[4] Topol EJ. High-performance medicine: the convergence of human and artificial intelligence Nat Med. 2019 25 44 56

[5] Haenssle H,Fink C,Toberer F.et al. Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions Ann Oncol. 2020 31 137 43

[6] Haenssle HA,Fink C,Schneiderbauer R.et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists Ann Oncol. 2018 29 1836 42

[7] Haenssle HA,Winkler JK,Fink C.et al. Skin lesions of face and scalp - classification by a market-approved convolutional neural network in comparison with 64 dermatologists Eur J Cancer. 2021 144 192 9

[8] Li M,Huang Z,Shan Q.et al. Performance and comparison of artificial intelligence and human experts in the detection and classification of colonic polyps BMC Gastroenterol. 2022 22 517 PMC9749329

[9] Loftus TJ,Vlaar AP,Hung AJ.et al. Executive summary of the artificial intelligence in surgery series Surgery. 2022 171 1435 9 PMC9379376

[10] De Backer P,Eckhoff JA,Simoens J.et al. Multicentric exploration of tool annotation in robotic surgery: lessons learned when starting a surgical artificial intelligence project Surg Endosc. 2022 36 8533 48

[11] Madad Zadeh S,Francois T,Calvet L.et al. SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology Surg Endosc. 2020 34 5377 83

[12] Ryu K,Kitaguchi D,Nakajima K.et al. Deep learning-based vessel automatic recognition for laparoscopic right hemicolectomy Surg Endosc. 2023 38 171 8

[13] Den Boer RB,De Jongh C,Huijbers WTE.et al. Computer-aided anatomy recognition in intrathoracic and -abdominal surgery: a systematic review Surg Endosc. 2022 36 8737 52 PMC9652273

[14] Zygomalas A,Kalles D,Katsiakis N,Anastasopoulos A,Skroubis G. Artificial intelligence assisted recognition of anatomical landmarks and laparoscopic instruments in transabdominal preperitoneal inguinal hernia repair Surg Innov. 2024 31 178 84

[15] Zhou R,Wang D,Zhang H.et al. Vision techniques for anatomical structures in laparoscopic surgery: a comprehensive review Front. Surg. 2025 12 1557153 PMC12034692

[16] Igaki T,Kitaguchi D,Kojima S.et al. Artificial intelligence-based total mesorectal excision plane navigation in laparoscopic colorectal surgery. Dis Colon Rectum. 2022;65:e329-e33

[17] Wang Z,Lu B,Long Y.et al. Autolaparo: a new dataset of integrated multi-tasks for image-guided surgical automation in laparoscopic hysterectomy. In: Wang L, Dou Q, Fletcher PT, Speidel S, Shuo Li S, Editors. International Conference on Medical Image Computing and Computer-Assisted Intervention; 2022 Sep 18-22; Singapore. Cham: Springer; 2022

[18] Narihiro S,Kitaguchi D,Hasegawa H,Takeshita N,Ito M. Deep learning-based real-time ureter identification in laparoscopic colorectal surgery Dis Colon Rectum. 2024 67 e1596 9

[19] Carstens M,Rinner FM,Bodenstedt S.et al. The dresden surgical anatomy dataset for abdominal organ segmentation in surgical data science Sci Data. 2023 10 3 PMC9837071

[20] Kamtam DN,Shrager JB,Malla SD.et al. Deep learning approaches to surgical video segmentation and object detection: a scoping review Comput Biol Med. 2025 194 110482

[21] Salort-Benejam L, Agudo A. NeRFscopy: neural radiance fields for in-vivo time-varying tissues from endoscopy. arXiv 2026;arXiv:2602.15775. Available from https://arxiv.org/abs/2401.00044 [accessed 27 May 2026].

[22] Chen H,Gou L,Fang Z.et al. Artificial intelligence assisted real-time recognition of intra-abdominal metastasis during laparoscopic gastric cancer surgery npj Digit. Med. 2025 8 9 PMC11701130

[23] Wang S,Li C,Wang R.et al. Annotation-efficient deep learning for automatic medical image segmentation Nat Commun. 2021 12 5915 PMC8501087

[24] Mascagni P,Alapatt D,Sestini L.et al. Computer vision in surgery: from potential to clinical value npj Digit. Med. 2022 5 163 PMC9616906

[25] Carstens M,Vasisht S,Zhang Z.et al. Artificial intelligence for surgical scene understanding: a systematic review and reporting quality meta-analysis. NPJ Digit Med. 2025;9:59 PMC12820105

[26] Meireles OR,Rosman G,Altieri MS.et al. ; SAGES Video Annotation for AI Working Groups. SAGES consensus recommendations on an annotation framework for surgical video Surg Endosc. 2021 35 4918 29

[27] Rädsch T,Reinke A,Weru V.et al. Labelling instructions matter in biomedical image analysis Nat Mach Intell. 2023 5 273 83

[28] Sener O, Savarese S. Active learning for convolutional neural networks: a core-set approach. arXiv 2017;arXiv:1708.00489. Available from https://doi.org/10.48550/arXiv.1708.00489 [accessed 27 May 2026].

[29] Wang CY,Yeh IH,Mark Liao HY. Yolov9: learning what you want to learn using programmable gradient information. In: Leonardis A, Ricci E, Roth S, Russakovsky O, Sattler T, Varol G, Editors. European Conference on Computer Vision 2024; 2024 Sep 29-Oct 4; Milan, Italy. Cham: Springer, 2024. pp. 1-21

[30] Zong Z, Song G, Liu Y. DETRs with collaborative hybrid assignments training. arXiv 2022;arXiv:2211.12860. Available from https://doi.org/10.48550/arXiv.2211.12860 [accessed 27 May 2026].

[31] Xie E, Wang W, Yu Z, Anandkumar A, Alvarez JM, Luo P. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv 2021;arXiv:2105.15203. Available from https://doi.org/10.48550/arXiv.2105.15203 [accessed 27 May 2026].

[32] Cheng B,Misra I,Schwing AG,Kirillov A,Girdhar R. Masked-attention mask transformer for universal image segmentation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2022 Jun 18-24; New Orleans, LA, USA. IEEE; 2022. pp. 1280-9

[33] van der Maaten L,Hinton G. Visualizing data using t-SNE. JMLR. 2008;9:2579-605. Available from: https://jmlr.org/papers/v9/vandermaaten08a.html. [Last accessed on 27 May 2026]

[34] Tan YZ,Cheng H,Ng KW,Ngiam KY,Gao Y,Khoo ET. Evaluating image matching with robust estimators: Bridging natural and surgical domains to enhance scene understanding. IEEE J Biomed Health Inform. 2025;29:8847-54

[35] Yang Z,Dai J,Pan J. 3D reconstruction from endoscopy images: a survey. Comput Biol Med. 2024;175:108546

[36] Zhao L,Wang T,Chen Y.et al. A novel framework for segmentation of small targets in medical images Sci Rep. 2025 15 9924 PMC11929788

[37] Yengera G, Mutter D, Marescaux J, Padoy N. Less is more: surgical phase recognition with less annotations through self-supervised pre-training of CNN-LSTM networks. arXiv 2018;arXiv:180508569. Available from https://doi.org/10.48550/arXiv.1805.08569 [accessed 27 May 2026].

[38] Raja MA,Loughran R,Mc Caffery F. NeuroEvolution of capsule networks for computer-aided laparoscopy. TechRxiv 2023. Available from https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.24648231.v1 [accessed 27 May 2026]

[39] Cui R,Zhang J,Pei J,Wang K,Heng PA,Qin J. Topology-constrained learning for efficient laparoscopic liver landmark detection. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2025; 2025 Sep 23-25; Daejeon, Republic of Korea. Cham: Springer; 2026. pp 585–94

[40] Peng Z,Wang Z,Yan Y.et al. Development of an AI-driven digital assistance system for real-time safety evaluation and quality control in laparoscopic liver surgery Front. Oncol. 2025 15 1678525 PMC12541588

[41] Tozzi F,Park HM,Mousavi SA.et al. Multimodal machine learning for staging laparoscopy: a combined image analysis and morphologic tool for the discrimination of peritoneal metastasis. Int J Surg. 2026;112:373-83 PMC12825761