Surgery faces a paradigm shift since it has developed rapidly in recent decades, becoming a high-tech discipline. Increasingly powerful technological developments such as modern operating rooms, featuring digital and interconnected equipment and novel imaging as well as robotic procedures, provide several data sources resulting in a huge potential to improve patient therapy and surgical outcome by means of Surgical Data Science. The emerging field of Surgical Data Science aims to improve the quality of surgery through acquisition, organization, analysis, and modeling of data, in particular using machine learning (ML). An integral part of surgical data science is to analyze the available data along the surgical treatment path and provide a context-aware autonomous action by means of ML methods. Autonomous actions related to surgical decision-making include preoperative decision support, intraoperative assistance functions, as well as robot-assisted actions. The goal is to democratize surgical skills and enhance the collaboration between surgeons and cyber-physical systems by quantifying surgical experience and making it accessible to machines, thereby improving patient therapy and outcome. The article introduces basic ML concepts as enablers for autonomous actions in surgery, highlighting examples for such actions along the surgical treatment path.

Background: The occurrence of biliary duct injury (BDI) after laparoscopic cholecystectomy (LC) remains 0.2-1.5%, which is largely caused by anatomic misidentifications. To solve this problem, we developed an artificial intelligence model, SurgSmart, and preliminarily verified its potential surgical guidance ability by comparing its performance with surgeons.

Methods: We prospectively collected 60 LC videos from November 2019 to August 2020 and enrolled 41 videos into the model establishment. Four important anatomic regions, namely cystic duct, cystic artery, common bile duct, and cystic plate, were annotated, and YOLOv3 (You Look Only Once), an object detection algorithm, was applied to develop the model SurgSmart. To further evaluate its performance, comparisons were made among SurgSmart, trainees, and seniors (surgical experience in LC > 100).

Results: In total, 101,863 frames were extracted from videos, and 5533 video frames were selected, annotated, and used in model training. The mean average precision (mAP) of SurgSmart was 0.710. Comparative results show SurgSmart had significantly higher intersection-over-union (IoU) and accuracy (IoU ≥ 0.5) in anatomy detection than those of seniors (n = 36) and trainees (n = 32) despite the existence of severe inflammation. Additionally, SurgSmart tended to correctly identify anatomic regions in earlier surgical phases than most of the seniors and trainees (P < 0.001).

Conclusions: SurgSmart is not only capable of accurately detecting and positioning anatomic regions in LC but also has better performance than that of the trainees and seniors in terms of individual still images and the whole set.

This white paper documents the consensus opinion of the expert members of the Editorial Board of Artificial Intelligence Surgery regarding the definitions of artificial intelligence and autonomy in regards to surgery and how the digital evolution of surgery is interrelated with the various forms of robotic-assisted surgery. It was derived from a series of video conference discussions, and the survey and results were subsequently revised and approved by all authors.

Aim: Hepatic steatosis is a recognised major risk factor for primary graft failure in liver transplantation. In general, the global fat burden is measured by the surgeon using a visual assessment. However, this can be augmented by a histological assessment, although there is often inter-observer variation in this regard as well. In many situations the assessment of the liver relies heavily on the experience of the observer and more experienced surgeons will accept organs that more junior surgeons feel are unsuitable for transplantation. Often surgeons will err on the side of caution and not accept a liver for fear of exposing recipients to excessive risk of death.

Methods: In this study, we present the use of deep learning for the non-invasive evaluation of donor liver organs. Transfer learning, using deep learning models such as the Visual Geometry Group (VGG) Face, VGG16, Residual Neural Network 50 (ResNet50), Dense Convolutional Network 121 (DenseNet121) and MobileNet are utilised for effective pattern extraction from partial and whole liver. Classification algorithms such as Support Vector Machines, k-Nearest Neighbour, Logistic Regression, Decision Tree and Linear Discriminant Analysis are then used for the final classification to identify between acceptable or non-acceptable donor liver organs.

Results: The proposed method is distinct in that we make use of image information both from partial and whole liver. We show that common pre-trained deep learning models can be used to quantify the donor liver steatosis with an accuracy of over 92%.

Conclusion: Machine learning algorithms offer the tantalising prospect of standardising the assessment and the possibility of using more donor organs for transplantation.