Artificial intelligence in laparoscopic cholecystectomy: does computer vision outperform human vision?

Abstract

Background: The occurrence of biliary duct injury (BDI) after laparoscopic cholecystectomy (LC) remains 0.2-1.5%, which is largely caused by anatomic misidentifications. To solve this problem, we developed an artificial intelligence model, SurgSmart, and preliminarily verified its potential surgical guidance ability by comparing its performance with surgeons.

Methods: We prospectively collected 60 LC videos from November 2019 to August 2020 and enrolled 41 videos into the model establishment. Four important anatomic regions, namely cystic duct, cystic artery, common bile duct, and cystic plate, were annotated, and YOLOv3 (You Look Only Once), an object detection algorithm, was applied to develop the model SurgSmart. To further evaluate its performance, comparisons were made among SurgSmart, trainees, and seniors (surgical experience in LC > 100).

Results: In total, 101,863 frames were extracted from videos, and 5533 video frames were selected, annotated, and used in model training. The mean average precision (mAP) of SurgSmart was 0.710. Comparative results show SurgSmart had significantly higher intersection-over-union (IoU) and accuracy (IoU ≥ 0.5) in anatomy detection than those of seniors (n = 36) and trainees (n = 32) despite the existence of severe inflammation. Additionally, SurgSmart tended to correctly identify anatomic regions in earlier surgical phases than most of the seniors and trainees (P < 0.001).

Conclusions: SurgSmart is not only capable of accurately detecting and positioning anatomic regions in LC but also has better performance than that of the trainees and seniors in terms of individual still images and the whole set.

Keywords

Laparoscopic cholecystectomy / artificial intelligence / deep learning / computer vision / artificial intelligence-surgeon comparation

References

[1] Soper NJ,Dunnegan DL.Laparoscopic cholecystectomy. The new “gold standard”?.Arch Surg1992;127:917-21; discussion 921

[2] Schwaitzberg SD,Jones DB.Threefold increased bile duct injury rate is associated with less surgeon experience in an insurance claims database: more rigorous training in biliary surgery may be needed.Surg Endosc2014;28:3068-73

[3] Törnqvist B,Akre O,Nilsson M.Selective intraoperative cholangiography and risk of bile duct injury during cholecystectomy.Br J Surg2015;102:952-8

[4] Barrett M,Chien HL,Telem DA.Bile duct injury and morbidity following cholecystectomy: a need for improvement.Surg Endosc2018;32:1683-8

[5] Brunt LM,Telem DA.the Prevention of Bile Duct Injury Consensus Work GroupSafe cholecystectomy multi-society practice guideline and state of the art consensus conference on prevention of bile duct injury during cholecystectomy.Ann Surg2020;272:3-23

[6] Lilley EJ,Jiang W.Intraoperative cholangiography during cholecystectomy among hospitalized medicare beneficiaries with non-neoplastic biliary disease.Am J Surg2017;214:682-6

[7] Törnqvist B,Persson G.Effect of intended intraoperative cholangiography and early detection of bile duct injury on survival after cholecystectomy: population based cohort study.BMJ2012;345:e6457 PMCID:PMC3469410

[8] Fong ZV,Strasberg SM.California Cholecystectomy GroupDiminished survival in patients with bile leak and ductal injury: management strategy and outcomes.J Am Coll Surg2018;226:568-576.e1 PMCID:PMC6053912

[9] Iwashita Y,Ohyama T.Delphi consensus on bile duct injuries during laparoscopic cholecystectomy: an evolutionary cul-de-sac or the birth pangs of a new technical framework?.J Hepatobiliary Pancreat Sci2017;24:591-602

[10] Esteva A,Ramsundar B.A guide to deep learning in healthcare.Nat Med2019;25:24-9

[11] LeCun Y,Hinton G.Deep learning.Nature2015;521:436-44

[12] Esteva A,Novoa RA.Dermatologist-level classification of skin cancer with deep neural networks.Nature2017;542:115-8 PMCID:PMC8382232

[13] Cicero M,Colak E.Training and validating a deep convolutional neural network for computer-aided detection and classification of abnormalities on frontal chest radiographs.Invest Radiol2017;52:281-7

[14] Gulshan V,Coram M.Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.JAMA2016;316:2402-10

[15] Rodriguez-Ruiz A,Gubern-Merida A.Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists.J Natl Cancer Inst2019;111:916-22 PMCID:PMC6748773

[16] Xie Y,Xia Y.Semi-supervised adversarial model for benign-malignant lung nodule classification on chest CT.Med Image Anal2019;57:237-48

[17] Grenda TR,Dimick JB.Using surgical video to improve technique and skill.Ann Surg2016;264:32-3 PMCID:PMC5671768

[18] Tan Mingxing, Le Quoc V. EfficientNet: rethinking model scaling for convolutional neural networks. 2019. Available from: http://proceedings.mlr.press/v97/tan19a/tan19a.pdf [Last accessed on 22 Apr 2022]

[19] Anteby R,Soffer S.Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis.Surg Endosc2021;35:1521-33

[20] Yamazaki Y,Matsuda T.Automated surgical instrument detection from laparoscopic gastrectomy video images using an open source convolutional neural network platform.J Am Coll Surg2020;230:725-732.e1

[21] Korndorffer JR Jr,Spain DA.Situating artificial intelligence in surgery: a focus on disease severity.Ann Surg2020;272:523-8

[22] Twinanda AP,Mutter D,Padoy N.RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations.IEEE Trans Med Imaging2019;38:1069-78

[23] Abdelrahman T,Egan R.Operative experience vs. competence: a curriculum concordance and learning curve analysis.J Surg Educ2016;73:694-8

[24] Marchi D,Gentile IG. Laparoscopic cholecystectomy: training, learning curve, and definition of expert. in: Agresta F, Campanile FC, Vettoretto N, editors. Laparoscopic cholecystectomy. Cham: Springer International Publishing; 2014. pp. 141-7.

[25] Maier-Hein L,Ross T.Heidelberg colorectal data set for surgical data science in the sensor operating room.Sci Data2021;8:101 PMCID:PMC8042116

[26] Liu C,Li S.ACF based region proposal extraction for YOLOv3 network towards high-performance cyclist detection in high resolution images.Sensors (Basel)2019;19:E2671 PMCID:PMC6630625

[27] Lin TY,Belongie S. , editors. Microsoft COCO: common objects in context 2014; Cham: Springer International Publishing.

[28] Madni TD,Minshall CT.The Parkland grading scale for cholecystitis.Am J Surg2018;215:625-30

[29] Movafegh F,Rassouli M,Nasiri M.Development and validation of the Iranian version of the patient privacy and confidentiality scale.Indian J Med Ethics2021;VI:1-13

[30] Kristen J. , Yoon-Soo P., Emil P. Are your assessment scores and feedback reliable? A statistical review for the surgical educator 2021. Available from: https://www.facs.org/education/division-of-education/publications/rise/articles/assessment-scores [Last accessed on 22 Apr 2022].

[31] Barisoni L,Hewitt SM,Balis UGJ.Digital pathology and computational image analysis in nephropathology.Nat Rev Nephrol2020;16:669-85 PMCID:PMC7447970

[32] Ruffle JK,Aziz Q.Artificial intelligence-assisted gastroenterology- promises and pitfalls.Am J Gastroenterol2019;114:422-8

[33] Madani A,Altieri MS.Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy.Ann Surg2020; PMCID:PMC8186165

[34] Sato M,Nakabayashi M.Computer vision for total laparoscopic hysterectomy.Asian J Endosc Surg2019;12:294-300

[35] Madad Zadeh S,Calvet L.SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology.Surg Endosc2020;34:5377-83

[36] Tokuyasu T,Matsunobu Y.Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy.Surg Endosc2021;35:1651-8 PMCID:PMC7940266

[37] Mascagni P,Alapatt D.Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning.Ann Surg2020;

[38] Hashimoto DA,Rus D.Artificial intelligence in surgery: promises and perils.Ann Surg2018;268:70-6 PMCID:PMC5995666