Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives

Abstract

Minimally invasive surgery, including laparoscopic and thoracoscopic procedures, benefits patients in terms of improved postoperative outcomes and short recovery time. The challenges in hand–eye coordination and manipulation dexterity during the aforementioned procedures have inspired an enormous wave of developments on surgical robotic systems to assist keyhole and endoscopic procedures in the past decades. This paper presents a systematic review of the state-of-the-art systems, picturing a detailed landscape of the system configurations, actuation schemes, and control approaches of the existing surgical robotic systems for keyhole and endoscopic procedures. The development challenges and future perspectives are discussed in depth to point out the need for new enabling technologies and inspire future researches.

Keywords

surgical robots / keyhole surgery / endoscopic surgery

References

[1] Cuschieri A. Laparoscopic surgery: current status, issues and future developments. Surgeon 2005; 3(3): 125–130, 132–133, 135–138

[2] Alimoglu O, Sagiroglu J, Atak I, Kilic A, Eren T, Caliskan M, Bas G. Robot-assisted laparoscopic (RAL) procedures in general surgery. Int J Med Robot 2016; 12(3): 427–430

[3] Taylor RH. A perspective on medical robotics. Proc IEEE 2006; 94(9): 1652–1664

[4] Kuo CH, Dai JS, Dasgupta P. Kinematic design considerations for minimally invasive surgical robots: an overview. Int J Med Robot 2012; 8(2): 127–145

[5] Bergeles C, Yang GZ. From passive tool holders to microsurgeons: safer, smaller, smarter surgical robots. IEEE Trans Biomed Eng 2014; 61(5): 1565–1576

[6] Smith JA, Jivraj J, Wong R, Yang V. 30 years of neurosurgical robots: review and trends for manipulators and associated navigational systems. Ann Biomed Eng 2016; 44(4): 836–846

[7] Jacofsky DJ, Allen M. Robotics in arthroplasty: a comprehensive review. J Arthroplasty 2016; 31(10): 2353–2363

[8] Kim C, Ryu SC, Dupont PE. Real-time adaptive kinematic model estimation of concentric tube robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015: 3214–3219

[9] Bergeles C, Gosline AH, Vasilyev NV, Codd PJ, Del Nido PJ, Dupont PE. Concentric tube robot design and optimization based on task and anatomical constraints. IEEE Trans Robot 2015; 31(1): 67–84

[10] Elhawary H, Tse ZTH, Hamed A, Rea M, Davies BL, Lamperth MU. The case for MR-compatible robotics: a review of the state of the art. Int J Med Robot 2008; 4(2): 105–113

[11] Burgner-Kahrs J, Rucker DC, Choset H. Continuum robots for medical applications: a survey. IEEE Trans Robot 2015; 31(6): 1261–1280

[12] Beasley RA. Medical robots: current systems and research directions. J Robot 2012; 2012: 1–14

[13] Diana M, Marescaux J. Robotic surgery. Br J Surg 2015; 102(2): e15–e28

[14] Rassweiler JJ, Autorino R, Klein J, Mottrie A, Goezen AS, Stolzenburg JU, Rha KH, Schurr M, Kaouk J, Patel V, Dasgupta P, Liatsikos E. Future of robotic surgery in urology. BJU Int 2017; 120(6): 822–841

[15] Navarra G, Pozza E, Occhionorelli S, Carcoforo P, Donini I. One-wound laparoscopic cholecystectomy. Br J Surg 1997; 84(5): 695

[16] Kalloo AN, Singh VK, Jagannath SB, Niiyama H, Hill SL, Vaughn CA, Magee CA, Kantsevoy SV. Flexible transgastric peritoneoscopy: a novel approach to diagnostic and therapeutic interventions in the peritoneal cavity. Gastrointest Endosc 2004; 60(1): 114–117

[17] Guthart GS, Salisbury JK. The Intuitive^TM Telesurgery System: overview and application. IEEE International Conference on Robotics and Automation (ICRA) 2000: 618–621

[18] Abdel Raheem A, Troya IS, Kim DK, Kim SH, Won PD, Joon PS, Hyun GS, Rha KH. Robot-assisted Fallopian tube transection and anastomosis using the new REVO-I robotic surgical system: feasibility in a chronic porcine model. BJU Int 2016; 118(4): 604–609

[19] Taylor R, Jensen P, Whitcomb L, Barnes A, Kumar R, Stoianovici D, Gupta P, Wang Z, Dejuan E, Kavoussi L. A steady-hand robotic system for microsurgical augmentation. Int J Robot Res 1999; 18(12): 1201–1210

[20] Li J, Zhou N, Wang S, Gao Y, Liu D. Design of an integrated master-slave robotic system for minimally invasive surgery. Int J Med Robot 2012; 8(1): 77–84

[21] Freschi C, Ferrari V, Melfi F, Ferrari M, Mosca F, Cuschieri A. Technical review of the da Vinci surgical telemanipulator. Int J Med Robot 2013; 9(4): 396–406

[22] Kim U, Lee DH, Kim YB, Seok DY, So J, Choi HR. S-Surge: novel portable surgical robot with multiaxis force-sensing capability for minimally invasive surgery. IEEE/ASME Trans Mechatron 2017; 22(4): 1717–1727

[23] Lum MJH, Friedman DCW, Sankaranarayanan G, King H, Fodero K, Leuschke R, Hannaford B, Rosen J, Sinanan MN. The RAVEN: design and validation of a telesurgery system. Int J Robot Res 2009; 28(9): 1183–1197

[24] Hannaford B, Rosen J, Friedman DW, King H, Roan P, Cheng L, Glozman D, Ma J, Kosari SN, White L. Raven-II: an open platform for surgical robotics research. IEEE Trans Biomed Eng 2013; 60(4): 954–959

[25] Kuo CH, Dai JS. Kinematics of a fully-decoupled remote center-of-motion parallel manipulator for minimally invasive surgery. J Med Device 2012; 6(2): 021008

[26] Mitsuishi M, Sugita N, Pitakwatchara P. Force feedback augmentation modes in the laparoscopic minimal invasive telesurgical system. IEEE/ASME Trans Mechatron 2007; 12(4): 447–454

[27] Berkelman P, Ma J. A compact modular teleoperated robotic system for laparoscopic surgery. Int J Robot Res 2009; 28(9): 1198–1215

[28] Dombre E, Michelin M, Pierrot F, Poignet P, Bidaud P, Morel G, Ortmaier T, Salle D, Zemiti N, Gravez P, Karouia M, Bonnet N. MARGE project: design, modelling, and control of assistive devices for minimally invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2004; LNCS 3217: 1–8

[29] Hagn U, Nickl M, Jörg S, Passig G, Bahls T, Nothhelfer A, Hacker F, Le-Tien L, Albu-Schäffer A, Konietschke R, Grebenstein M, Warpup R, Haslinger R, Frommberger M, Hirzinger G. The DLR MIRO: a versatile lightweight robot for surgical applications. Industrial Robot 2008; 35(4): 324–336

[30] Lopez E, Kwok KW, Payne CJ, Giataganas P, Yang GZ. Implicit active constraints for robot-assisted arthroscopy. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5390–5395

[31] Dai Z, Wu Z, Zhao J, Xu K. A robotic laparoscopic tool with enhanced capabilities and modular actuation. Sci China Technol Sci 2019; 62(1): 47–59

[32] Çavuşoğlu MC, Cohn M, Tendick F, Sastry S. A laparoscopic telesurgical workstation. IEEE Trans Robot Autom 1999; 15(4): 728–739

[33] Sekimoto M, Nishikawa A, Taniguchi K, Takiguchi S, Miyazaki F, Doki Y, Mori M. Development of a compact laparoscope manipulator (P-arm). Surg Endosc 2009; 23(11): 2596–2604

[34] Dalvand MM, Shirinzadeh B. Remote centre-of-motion control algorithms of 6-RRCRR parallel robot assisted surgery system (PRAMiSS). IEEE International Conference on Robotics and Automation (ICRA) 2012: 3401–3406

[35] Çavuşoğlu MC, Williams W, Tendick F, Sastry SS. Robotics for telesurgery: second generation Berkeley/UCSF laparoscopic telesurgical workstation and looking towards the future applications. Industrial Robot 2003; 30(1): 22–29

[36] Nasseri MA, Gschirr P, Eder M, Nair S, Kobuch K, Maier M, Zapp D, Lohmann C, Knoll A. Virtual fixture control of a hybrid parallel-serial robot for assisting ophthalmic surgery: an experimental study. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2014: 732–738

[37] Azimian H, Patel RV, Naish MD. On constrained manipulation in robotics-assisted minimally invasive surgery. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2010: 650–655

[38] Sandoval J, Poisson G, Vieyres P. A new kinematic formulation of the RCM constraint for redundant torque-controlled robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017: 4576–4581

[39] Funda J, Taylor RH, Eldridge B, Gomory S, Gruben KG. Constrained cartesian motion control for teleoperated surgical robots. IEEE Trans Robot Autom 1996; 12(3): 453–465

[40] Locke RCO, Patel RV. Optimal remote center-of-motion location for robotics-assisted minimally-invasive surgery. IEEE International Conference on Robotics and Automation (ICRA) 2007: 1900–1905

[41] Aghakhani N, Geravand M, Shahriari N, Vendittelli M, Oriolo G. Task control with remote center of motion constraint for minimally invasive robotic surgery. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5807–5812

[42] Michelin M, Poignet P, Dombre E. Dynamic task/posture decoupling for minimally invasive surgery motions: simulation results. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2004: 3625–3630

[43] Cha HJ, Yi BJ. Modeling of a constraint force at RCM point in a needle insertion task. IEEE International Conference on Mechatronics and Automation (ICMA) 2011: 2177–2182

[44] Taylor RH, Funda J, Eldridge B, Gomory S, Gruben K, LaRose D, Talamini M, Kavoussi L, Anderson J. A telerobotics assistant for laparoscopic surgery. IEEE Eng Med Biol Mag 1995; 14(3): 279–288

[45] Bunter SE, Ghodoussi M. Transforming a surgical robot for human telesurgery. IEEE Trans Robot Autom 2003; 19(5): 818–824

[46] Wang W, Li J, Wang S, Su H, Jiang X. System design and animal experiment study of a novel minimally invasive surgical robot. Int J Med Robot 2016; 12(1): 73–84

[47] Ikuta K, Daifu S, Hasegawa T, Higashikawa H. Hyper-finger for remote minimally invasive surgery in deep area. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2002; 2488: 173–181

[48] Dachs GW, Peine WJ. A novel surgical robot design: minimizing the operating envelope within the sterile field. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS) 2006: 1505–1508

[49] Nakamura Y, Matsui A, Saito T, Yoshimoto K. Shape-memory-alloy active forceps for laparoscopic surgery. IEEE International Conference on Robotics and Automation (ICRA) 1995: 2320–2327

[50] Shi ZY, Liu D, Wang TM. A shape memory alloy-actuated surgical instrument with compact volume. Int J Med Robot 2014; 10(4): 474–481

[51] Takahashi H, Warisawa S, Mitsuishi M, Arata J, Hashizume M. Development of high dexterity minimally invasive surgical system with augmented force feedback capability. IEEE / RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2006: 284–289

[52] Hong MB, Jo YH. Design of a novel 4-DOF wrist-type surgical instrument with enhanced rigidity and dexterity. IEEE/ASME Trans Mechatron 2014; 19(2): 500–511

[53] Robinson G, Davies JB. Continuum robots—a state of the art. IEEE International Conference on Robotics and Automation (ICRA) 1999: 2849–2853

[54] Francis P, Eastwood KW, Bodani V, Price K, Upadhyaya K, Podolsky D, Azimian H, Looi T, Drake J. Miniaturized instruments for the da Vinci research kit: design and implementation of custom continuum tools. IEEE Robot Autom Mag 2017; 24(2): 24–33

[55] Abbott DJ, Becke C, Rothstein RI, Peine WJ. Design of an endoluminal NOTES robotic system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2007: 410–416

[56] Kaouk JH, Haber GP, Autorino R, Crouzet S, Ouzzane A, Flamand V, Villers A. A novel robotic system for single-port urologic surgery: first clinical investigation. Eur Urol 2014; 66(6): 1033–1043

[57] Xu K, Zhao J, Fu M. Development of the SJTU unfoldable robotic system (SURS) for single port laparoscopy. IEEE/ASME Trans Mechatron 2015; 20(5): 2133–2145

[58] Arata J, Fujisawa Y, Nakadate R, Kiguchi K, Harada K, Mitsuishi M, Hashizume M. Compliant four degree-of-freedom manipulator with locally deformable elastic elements for minimally invasive surgery. IEEE International Conference on Robotics and Automation (ICRA) 2019: 2663–2669

[59] Tobushi H, Furuichi Y, Sakuragi T, Sugimoto Y. Bending fatigue properties of a superelastic thin tube and a high-elastic thin wire of TiNi alloy. Mater Trans 2009; 50(8): 2043–2049

[60] Vaida C, Plitea N, Pisla D, Gherman B. Orientation module for surgical instruments—a systematical approach. Meccanica 2013; 48(1): 145–158

[61] Kanno T, Haraguchi D, Yamamoto M, Tadano K, Kawashima K. A forceps manipulator with flexible 4-DOF mechanism for laparoscopic surgery. IEEE/ASME Trans Mechatron 2015; 20(3): 1170–1178

[62] Sa Z, Chen Y, Li Q, Zhao B, Xu K. Kinematic optimization of a continuum surgical manipulator. IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018: 2069–2074

[63] Wu Z, Li Q, Zhao J, Gao J, Xu K. Design of a modular continuum-articulated laparoscopic robotic tool with decoupled kinematics. IEEE Robot Autom Lett 2019; 4(4): 3545–3552

[64] Morelli L, Guadagni S, Di Franco G, Palmeri M, Di Candio G, Mosca F. Da Vinci single site© surgical platform in clinical practice: a systematic review. Int J Med Robot 2016; 12(4): 724–734

[65] Choi H, Kwak HS, Lim YA, Kim HJ. Surgical robot for single-incision laparoscopic surgery. IEEE Trans Biomed Eng 2014; 61(9): 2458–2466

[66] Seeliger B, Diana M, Ruurda JP, Konstantinidis KM, Marescaux J, Swanström LL. Enabling single-site laparoscopy: the SPORT platform. Surg Endosc 2019; 33(11): 3696–3703

[67] Walker AS, Steele SR. The future of robotic instruments in colon and rectal surgery. Semin Colon Rectal Surg 2016; 27(3): 144–149

[68] Roh KS, Yoon S, Do Kwon Y, Shim Y, Kim YJ. Single-port surgical robot system with flexible surgical instruments. International Conference on Intelligent Robotics and Applications (ICIRA) 2015: 447–459

[69] Petroni G, Niccolini M, Menciassi A, Dario P, Cuschieri A. A novel intracorporeal assembling robotic system for single-port laparoscopic surgery. Surg Endosc 2013; 27(2): 665–670

[70] Wortman TD, Mondry JM, Farritor SM, Oleynikov D. Single-site colectomy with miniature in vivo robotic platform. IEEE Trans Biomed Eng 2013; 60(4): 926–929

[71] Yung KL, Cheung JLK, Chung SW, Singh S, Yeung CK. A single-port robotic platform for laparoscopic surgery with a large central channel for additional instrument. Ann Biomed Eng 2017; 45(9): 2211–2221

[72] Kobayashi Y, Sekiguchi Y, Noguchi T, Takahashi Y, Liu Q, Oguri S, Toyoda K, Uemura M, Ieiri S, Tomikawa M, Ohdaira T, Hashizume M, Fujie MG. Development of a robotic system with six-degrees-of-freedom robotic tool manipulators for single-port surgery. Int J Med Robot 2015; 11(2): 235–246

[73] Cheon B, Gezgin E, Ji DK, Tomikawa M, Hashizume M, Kim HJ, Hong J. A single port laparoscopic surgery robot with high force transmission and a large workspace. Surg Endosc 2014; 28(9): 2719–2729

[74] Xu K, Goldman RE, Ding J, Allen PK, Fowler DL, Simaan N. System design of an insertable robotic effector platform for single port access (SPA) surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2009: 5546–5552

[75] Wang H, Zhang R, Chen W, Wang X, Pfeifer R. A cable-driven soft robot surgical system for cardiothoracic endoscopic surgery: preclinical tests in animals. Surg Endosc 2017; 31(8): 3152–3158

[76] Zhao J, Feng B, Zheng MH, Xu K. Surgical robots for SPL and NOTES: a review. Minim Invasive Ther Allied Technol 2015; 24(1): 8–17

[77] Mintz Y, Horgan S, Cullen J, Stuart D, Falor E, Talamini MA. NOTES: a review of the technical problems encountered and their solutions. J Laparoendosc Adv Surg Tech A 2008; 18(4): 583–587

[78] Ikuta K, Yamamoto K, Sasaki K. Development of remote microsurgery robot and new surgical procedure for deep and narrow space. IEEE International Conference on Robotics and Automation (ICRA) 2003: 1103–1108

[79] Phee SJ, Low SC, Sun ZL, Ho KY, Huang WM, Thant ZM. Robotic system for no-scar gastrointestinal surgery. Int J Med Robot 2008; 4(1): 15–22

[80] Lehman AC, Dumpert J, Wood NA, Redden L, Visty AQ, Farritor S, Varnell B, Oleynikov D. Natural orifice cholecystectomy using a miniature robot. Surg Endosc 2009; 23(2): 260–266

[81] Tortora G, Dario P, Menciassi A. Array of robots augmenting the kinematics of endocavitary surgery. IEEE/ASME Trans Mechatron 2014; 19(6): 1821–1829

[82] Son J, Cho CN, Kim KG, Chang TY, Jung H, Kim SC, Kim MT, Yang N, Kim TY, Sohn DK. A novel semi-automatic snake robot for natural orifice transluminal endoscopic surgery: preclinical tests in animal and human cadaver models (with video). Surg Endosc 2015; 29(6): 1643–1647

[83] Clark J, Noonan DP, Vitiello V, Sodergren MH, Shang J, Payne CJ, Cundy TP, Yang GZ, Darzi A. A novel flexible hyper-redundant surgical robot: prototype evaluation using a single incision flexible access pelvic application as a clinical exemplar. Surg Endosc 2015; 29(3): 658–667

[84] Zhao J, Zheng X, Zheng M, Shih AJ, Xu K. An endoscopic continuum testbed for finalizing system characteristics of a surgical robot for NOTES procedures. IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2013: 63–70

[85] Goldman RE, Bajo A, MacLachlan LS, Pickens R, Herrell SD, Simaan N. Design and performance evaluation of a minimally invasive telerobotic platform for transurethral surveillance and intervention. IEEE Trans Biomed Eng 2013; 60(4): 918–925

[86] Burgner J, Rucker DC, Gilbert HB, Swaney PJ, Russell PT 3rd, Weaver KD, Webster RJ 3rd. A telerobotic system for transnasal surgery. IEEE/ASME Trans Mechatron 2014; 19(3): 996–1006

[87] Thakkar S, Awad M, Gurram KC, Tully S, Wright C, Sanan S, Choset H. A novel, new robotic platform for natural orifice distal pancreatectomy. Surg Innov 2015; 22(3): 274–282

[88] Mitchell CR, Hendrick RJ, Webster RJ 3rd, Herrell SD. Toward improving transurethral prostate surgery: development and initial experiments with a prototype concentric tube robotic platform. J Endourol 2016; 30(6): 692–696

[89] Légner A, Diana M, Halvax P, Liu YY, Zorn L, Zanne P, Nageotte F, De Mathelin M, Dallemagne B, Marescaux J. Endoluminal surgical triangulation 2.0: a new flexible surgical robot. Preliminary pre-clinical results with colonic submucosal dissection. Int J Med Robot 2017; 13(3): e1819

[90] Xu K, Liang B, Dai Z, Zhao J, Zhao B, Liu H, Xiao L, Sun Y. Preliminary development of a continuum dual-arm surgical robotic system for transurethral procedures. International Conference on Intelligent Robotics and Applications (ICIRA) 2017: 311–322

[91] Wagner CR, Howe RD. Force feedback benefit depends on experience in multiple degree of freedom robotic surgery task. IEEE Trans Robot Autom 2007; 23(6): 1235–1240

[92] Wagner CR, Stylopoulos N, Howe RD. The role of force feedback in surgery: analysis of blunt dissection. Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS) 2002: 68–74

[93] Xu K, Simaan N. An investigation of the intrinsic force sensing capabilities of continuum robots. IEEE Trans Robot 2008; 24(3): 576–587

[94] Hong MB, Jo YH. Design and evaluation of 2-DOF compliant forceps with force-sensing capability for minimally invasive robot surgery. IEEE Trans Robot 2012; 28(4): 932–941

[95] Sokhanvar S, Packirisamy M, Dargahi J. A multifunctional PVDF-based tactile sensor for minimally invasive surgery. Smart Mater Struct 2007; 16(4): 989–998

[96] Zarrin PS, Escoto A, Xu R, Patel RV, Naish MD, Trejos AL. Development of an optical fiber-based sensor for grasping and axial force sensing. IEEE International Conference on Robotics and Automation (ICRA) 2017: 939–944

[97] Kim U, Kim YB, So J, Seok D, Choi HR. Sensorized surgical forceps for robotic-assisted minimally invasive surgery. IEEE Trans Ind Electron 2018; 65(12): 9604–9613

[98] He C, Wang S, Sang H, Li J, Zhang L. Force sensing of multiple-DOF cable-driven instruments for minimally invasive robotic surgery. Int J Med Robot 2014; 10(3): 314–324

[99] Lee D, Kim U, Gulrez T, Yoon WJ, Hannaford B, Choi HR. A laparoscopic grasping tool with force sensing capability. IEEE/ASME Trans Mechatron 2016; 21(1): 130–141

[100] Tholey G, Desai JP, Castellanos AE. Force feedback plays a significant role in minimally invasive surgery results and analysis. Ann Surg 2005; 241(1): 102–109

[101] Seibold U, Kubler B, Hirzinger G. Prototype of instrument for minimally invasive surgery with 6-axis force sensing capability. IEEE International Conference on Robotics and Automation (ICRA) 2005: 496–501

[102] Moradi Dalvand M, Shirinzadeh B, Shamdani AH, Smith J, Zhong Y. An actuated force feedback-enabled laparoscopic instrument for robotic-assisted surgery. Int J Med Robot 2014; 10(1): 11–21

[103] Peirs J, Clijnen J, Reynaerts D, Brussel HV, Herijgers P, Corteville B, Boone S. A micro optical force sensor for force feedback during minimally invasive robotic surgery. Sens Actuators A Phys 2004; 115(2): 447–455

[104] Noh Y, Sareh S, Würdemann H, Liu H, Back J, Housden J, Rhode K, Althoefer K. Three-axis fiber-optic body force sensor for flexible manipulators. IEEE Sens J 2016; 16(6): 1641–1651

[105] Haslinger R, Leyendecker P, Seibold U. A fiberoptic force-torque-sensor for minimally invasive robotic surgery. 2013 IEEE International Conference on Robotics and Automation 2013: 4390–4395

[106] Qasaimeh MA, Sokhanvar S, Dargahi J, Kahrizi M. PVDF-based microfabricated tactile sensor for minimally invasive surgery. J Microelectromech Syst 2009; 18(1): 195–207

[107] Xu K, Simaan N. Intrinsic wrench estimation and its performance index of multi-segment continuum robots. IEEE Trans Robot 2010; 26(3): 555–561

[108] Lee SJ, Lee SC, Ahn HS. Design and control of tele-matched surgery robot. Mechatronics 2014; 24(5): 395–406

[109] Kuebler B, Seibold U, Hirzinger G. Development of actuated and sensor integrated forceps for minimally invasive robotic surger. Int J Med Robot 2005; 1(3): 96–107

[110] Baumhauer M, Feuerstein M, Meinzer HP, Rassweiler J. Navigation in endoscopic soft tissue surgery: perspectives and limitations. J Endourol 2008; 22(4): 751–766

[111] Schols RM, Bouvy ND, van Dam RM, Stassen LP. Advanced intraoperative imaging methods for laparoscopic anatomy navigation: an overview. Surg Endosc 2013; 27(6): 1851–1859

[112] Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang GZ. Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2005: 139–146

[113] Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 2009; 73(4): 896–900

[114] Röhl S, Bodenstedt S, Suwelack S, Dillmann R, Speidel S, Kenngott H, Muller-Stich BP. Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 2012; 39(3): 1632–1645

[115] Clancy NT, Stoyanov D, Maier-Hein L, Groch A, Yang GZ, Elson DS. Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging. Biomed Opt Express 2011; 2(11): 3119–3128

[116] Schmalz C, Forster F, Schick A, Angelopoulou E. An endoscopic 3D scanner based on structured light. Med Image Anal 2012; 16(5): 1063–1072

[117] Le HND, Nguyen H, Wang Z, Opfermann J, Leonard S, Krieger A, Kang JU. Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis. J Biomed Opt 2018; 23(5): 1–10

[118] Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D. Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 2013; 17(8): 974–996

[119] Bartoli A, Gerard Y, Chadebecq F, Collins T. On template-based reconstruction from a single view: analytical solutions and proofs of wellposedness for developable, isometric and conformal surfaces. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012: 2026–2033

[120] Wu C, Narasimhan SG, Jaramaz B. A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int J Comput Vis 2010; 86(2–3): 211–228

[121] Mountney P, Stoyanov D, Davison A, Yang GZ. Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2006: 347–354

[122] Mountney P, Yang GZ. Motion compensated SLAM for image guided surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2010: 496–504

[123] Mahmoud N, Cirauqui I, Hostettler A, Doignon C, Soler L, Marescaux J, Montiel JMM. ORBSLAM-based endoscope tracking and 3D reconstruction. International Workshop on Computer-Assisted and Robotic Endoscopy (CARE) 2016: 72–83

[124] Song J, Wang J, Zhao L, Huang S, Dissanayake G. MIS-SLAM: real-time large-scale dense deformable SLAM system in minimal invasive surgery based on heterogeneous computing. IEEE Robot Autom Lett 2018; 3(4): 4068–4075

[125] Daskalaki D, Aguilera F, Patton K, Giulianotti PC. Fluorescence in robotic surgery. J Surg Oncol 2015; 112(3): 250–256

[126] Regeling B, Thies B, Gerstner AOH, Westermann S, Müller NA, Bendix J, Laffers W. Hyperspectral imaging using flexible endoscopy for laryngeal cancer detection. Sensors (Basel) 2016; 16(8): E1288

[127] Erden MS, Rosa B, Boularot N, Gayet B, Morel G, Szewczyk J. Conic-spiraleur: a miniature distal scanner for confocal microlaparoscope. IEEE/ASME Trans Mechatron 2014; 19(6): 1786–1798

[128] Frangioni JV. In vivo near-infrared fluorescence imaging. Curr Opin Chem Biol 2003; 7(5): 626–634

[129] Tobis S, Knopf J, Silvers C, Yao J, Rashid H, Wu G, Golijanin D. Near infrared fluorescence imaging with robotic assisted laparoscopic partial nephrectomy: initial clinical experience for renal cortical tumors. J Urol 2011; 186(1): 47–52

[130] Bjurlin MA, Gan M, McClintock TR, Volpe A, Borofsky MS, Mottrie A, Stifelman MD. Near-infrared fluorescence imaging: emerging applications in robotic upper urinary tract surgery. Eur Urol 2014; 65(4): 793–801

[131] Spinoglio G, Priora F, Bianchi PP, Lucido FS, Licciardello A, Maglione V, Grosso F, Quarati R, Ravazzoni F, Lenti LM. Real-time near-infrared (NIR) fluorescent cholangiography in single-site robotic cholecystectomy (SSRC): a single-institutional prospective study. Surg Endosc 2013; 27(6): 2156–2162

[132] Wagner OJ, Louie BE, Vallières E, Aye RW, Farivar AS. Near-infrared fluorescence imaging can help identify the contralateral phrenic nerve during robotic thymectomy. Ann Thorac Surg 2012; 94(2): 622–625

[133] Holloway RW, Bravo RAM, Rakowski JA, James JA, Jeppson CN, Ingersoll SB, Ahmad S. Detection of sentinel lymph nodes in patients with endometrial cancer undergoing robotic-assisted staging: a comparison of colorimetric and fluorescence imaging. Gynecol Oncol 2012; 126(1): 25–29

[134] Jafari MD, Lee KH, Halabi WJ, Mills SD, Carmichael JC, Stamos MJ, Pigazzi A. The use of indocyanine green fluorescence to assess anastomotic perfusion during robotic assisted laparoscopic rectal surgery. Surg Endosc 2013; 27(8): 3003–3008

[135] Decker RS, Shademan A, Opfermann JD, Leonard S, Kim PCW, Krieger A. Biocompatible near-infrared three-dimensional tracking system. IEEE Trans Biomed Eng 2017; 64(3): 549–556

[136] Li Q, He X, Wang Y, Liu H, Xu D, Guo F. Review of spectral imaging technology in biomedical engineering: achievements and challenges. J Biomed Opt 2013; 18(10): 100901

[137] Stoyanov D. Surgical vision. Ann Biomed Eng 2012; 40(2): 332–345

[138] Lian J, Zheng Y, Jiao W, Yan F, Zhao B. Deblurring sequential ocular images from multi-spectral imaging (MSI) via mutual information. Med Biol Eng Comput 2018; 56(6): 1107–1113

[139] Olweny EO, Faddegon S, Best SL, Jackson N, Wehner EF, Tan YK, Zuzak KJ, Cadeddu JA. Renal oxygenation during robot-assisted laparoscopic partial nephrectomy: characterization using laparoscopic digital light processing hyperspectral imaging. J Endourol 2013; 27(3): 265–269

[140] Moccia S, Wirkert SJ, Kenngott H, Vemuri AS, Apitz M, Mayer B, De Momi E, Mattos LS, Maier-Hein L. Uncertainty-aware organ classification for surgical data science applications in laparoscopy. IEEE Trans Biomed Eng 2018; 65(11): 2649–2659

[141] Gmitro AF, Aziz D. Confocal microscopy through a fiber-optic imaging bundle. Opt Lett 1993; 18(8): 565–567

[142] Zuo S, Yang GZ. Endomicroscopy for computer and robot assisted intervention. IEEE Rev Biomed Eng 2017; 10: 12–25

[143] Zuo S, Hughes M, Seneci C, Chang TP, Yang GZ. Toward intraoperative breast endomicroscopy with a novel surface-scanning device. IEEE Trans Biomed Eng 2015; 62(12): 2941–2952

[144] Giataganas P, Hughes M, Yang GZ. Force adaptive robotically assisted endomicroscopy for intraoperative tumour identification. Int J CARS 2015; 10(6): 825–832

[145] Rosa B, Erden MS, Vercauteren T, Herman B, Szewczyk J, Morel G. Building large mosaics of confocal edomicroscopic images using visual servoing. IEEE Trans Biomed Eng 2013; 60(4): 1041–1049

[146] Giataganas P, Vitiello V, Simaiaki V, Lopez E, Yang GZ. Cooperative in situ microscopic scanning and simultaneous tissue surface reconstruction using a compliant robotic manipulator. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5378–5383

[147] Mountney P, Giannarou S, Elson D, Yang GZ. Optical biopsy mapping for minimally invasive cancer screening. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2009: 483–490

[148] Avgousti S, Christoforou EG, Panayides AS, Voskarides S, Novales C, Nouaille L, Pattichis CS, Vieyres P. Medical telerobotic systems: current status and future trends. Biomed Eng Online 2016; 15(1): 96

[149] Yang GZ, Cambias J, Cleary K, Daimler E, Drake J, Dupont PE, Hata N, Kazanzides P, Martel S, Patel RV, Santos VJ, Taylor RH. Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci Robot 2017; 2(4): eaam8638

[150] Yip M, Das N. Robot autonomy for surgery. The Encyclopedia of Medical Robotics. World Scientific Publishing, 2018: 281–313

[151] Rovetta A, Sala R, Cosmi F, Wen X, Sabbadini D, Milanesi S, Togno A, Angelini L, Bejczy A. The first experiment in the world of robotic telesurgery for laparoscopy carried out by means of satellites networks and optical fibres networks on 7th July 1993. International Conference on Industrial Electronics, Control, and Instrumentation (IECON) 1993; 1: 51–56

[152] Okamura AM. Methods for haptic feedback in teleoperated robot-assisted surgery. Ind Rob 2004; 31(6): 499–508

[153] Talasaz A, Trejos AL, Patel RV. Effect of force feedback on performance of robotics-assisted suturing. IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2012: 823–828

[154] Birglen L, Gosselin C, Pouliot N, Monsarrat B, Laliberté T. SHaDe, a new 3-DOF haptic device. IEEE Trans Robot Autom 2002; 18(2): 166–175

[155] Steger R, Lin K, Adelstein BD, Kazerooni H. Design of a passively balanced spatial linkage haptic interface. J Mech Des 2004; 126(6): 984–991

[156] Arata J, Kondo H, Ikedo N, Fujimoto H. Haptic device using a newly developed redundant parallel mechanism. IEEE Trans Robot 2011; 27(2): 201–214

[157] Tholey G, Desai JPA. General-purpose 7 DOF haptic device: applications toward robot-assisted surgery. IEEE/ASME Trans Mechatron 2007; 12(6): 662–669

[158] Stocco LJ, Salcudean SE, Sassani F. Optimal kinematic design of a haptic pen. IEEE/ASME Trans Mechatron 2001; 6(3): 210–220

[159] Ueberle M, Buss M. Design, control, and evaluation of a new 6 DOF haptic device. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2002: 2949–2954

[160] Kim K, Kyun W. Design and analysis of a new 7-DOF parallel type haptic device: PATHOS-II. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2003: 2241–2246

[161] Faulring EL, Colgate JE, Peshkin MA. The cobotic hand controller: design, control and performance of a novel haptic display. Int J Robot Res 2006; 25(11): 1099–1119

[162] Najdovski Z, Nahavandi S, Fukuda T. Design, development, and evaluation of a pinch–grasp haptic interface. IEEE/ASME Trans Mechatron 2014; 19(1): 45–54

[163] Lee G, Hur SM, Oh Y. A novel haptic device with high-force display capability and wide workspace. IEEE International Conference on Robotics and Automation (ICRA) 2016: 2704–2709

[164] Zhao B, Sa Z, Wu Z, Li Q, Xu K. CombX: design of a haptic device for teleoperation. IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018: 1969–1974

[165] Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. Automatica 2006; 42(12): 2035–2057

[166] Hashtrudi-Zaad K, Salcudean SE. Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int J Robot Res 2001; 20(6): 419–445

[167] Lawrence DA. Stability and transparency in bilateral teleoperation. IEEE Trans Robot Autom 1993; 9(5): 624–637

[168] Tavakoli M, Aziminejad A, Patel RV, Moallem M. High-fidelity bilateral teleoperation systems and the effect of multimodal haptics. IEEE Trans Syst Man Cybern B Cybern 2007; 37(6): 1512–1528

[169] Mahvash M, Okamura A. Friction compensation for enhancing transparency of a teleoperator with compliant transmission. IEEE Trans Robot 2007; 23(6): 1240–1246

[170] Torabi A, Khadem M, Zareinia K, Sutherland GR, Tavakoli M. Application of a redundant haptic interface in enhancing soft-tissue stiffness discrimination. IEEE Robot Autom Lett 2019; 4(2): 1037–1044

[171] Preusche C, Ortmaier T, Hirzinger G. Teleoperation concepts in minimal invasive surgery. Control Eng Pract 2002; 10(11): 1245–1250

[172] Tavakoli M, Patel RV, Moallem M. Haptic interaction in robot-assisted endoscopic surgery: a sensorized end-effector. Int J Med Robot 2005; 1(2): 53–63

[173] Kim U, Seok DY, Kim YB, Lee DH, Choi HR. Development of a grasping force-feedback user interface for surgical robot system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016: 845–850

[174] Cavusoglu MC, Sherman A, Tendick F. Design of bilateral teleoperation controllers for haptic exploration and telemanipulation of soft environments. IEEE Trans Robot Autom 2002; 18(4): 641–647

[175] Tavakoli M, Howe RD. Haptic effects of surgical teleoperator flexibility. Int J Robot Res 2009; 28(10): 1289–1302

[176] Rosenberg LB. Virtual fixtures: perceptual tools for telerobotic manipulation. Proceedings of IEEE Virtual Reality Annual International Symposium 1993: 76–82

[177] Abbott JJ, Marayong P, Okamura AM. Haptic virtual fixtures for robot-assisted manipulation. Robotics Research 2007; 28: 49–64

[178] Lopez E, Zollo L, Guglielmelli E. Teleoperated control based on virtual fixtures for a redundant surgical system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013: 450–455

[179] Kapoor A, Li M, Taylor RH. Spatial motion constraints for robot assisted suturing using virtual fixtures. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2005: 89–96

[180] Shahbazi M, Atashzar SF, Patel RV. A dual-user teleoperated system with virtual fixtures for robotic surgical training. IEEE International Conference on Robotics and Automation (ICRA) 2013: 3639–3644

[181] Vitrani MA, Poquet C, Morel G. Applying virtual fixtures to the distal end of a minimally invasive surgery instrument. IEEE Trans Robot 2017; 33(1): 114–123

[182] Chen Z, Malpani A, Chalasani P, Deguet A, Vedula SS, Kazanzides P, Taylor RH. Virtual fixture assistance for needle passing and knot tying. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016: 2343–2350

[183] Fontanelli GA, Yang GZ, Siciliano B. A comparison of assistive methods for suturing in MIRS. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2018: 4389–4395

[184] Haidegger T. Autonomy for surgical robots: concepts and paradigms. IEEE Transactions on Medical Robotics and Bionics 2019; 1(2): 65–76

[185] Moustris GP, Hiridis SC, Deliparaschos KM, Konstantinidis KM. Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. Int J Med Robot 2011; 7(4): 375–392

[186] Hoeckelmann M, Rudas IJ, Fiorini P, Kirchne F, Haidegger T. Current capabilities and development potential in surgical robotics. Int J Adv Robot Syst 2015; 12(5): 61

[187] Münzer B, Schoeffmann K, Böszörmenyi L. Content-based processing and analysis of endoscopic images and videos: a survey. Multimedia Tools Appl 2018; 77(1): 1323–1362

[188] Wei GQ, Arbter K, Hirzinger G. Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation. IEEE Eng Med Biol Mag 1997; 16(1): 40–45

[189] Voros S, Long JA, Cinquin P. Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res 2007; 26(11–12): 1173–1190

[190] Pezzementi Z, Voros S, Hager GD. Articulated object tracking by rendering consistent appearance parts. IEEE International Conference on Robotics and Automation (ICRA) 2009: 3940–3947

[191] Kranzfelder M, Schneider A, Fiolka A, Schwan E, Gillen S, Wilhelm D, Schirren R, Reiser S, Jensen B, Feussner H. Real-time instrument detection in minimally invasive surgery using radiofrequency identification technology. J Surg Res 2013; 185(2): 704–710

[192] Krupa A, Gangloff J, Doignon C, de Mathelin MF, Morel G, Leroy J, Soler L, Marescaux J. Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans Robot Autom 2003; 19(5): 842–853

[193] Nageotte F, Zanne P, Doignon C, de Mathelin M. Visual servoing-based endoscopic path following for robot-assisted laparoscopic surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2006: 2364–2369

[194] Reiter A, Allen PK, Zhao T. Feature classification for tracking articulated surgical tools. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2012: 592–600

[195] Ye M, Zhang L, Giannarou S, Yang GZ. Real-time 3D tracking of articulated tools for robotic surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2016: 386–394

[196] Wolf R, Duchateau J, Cinquin P, Voros S. 3D tracking of laparoscopic instruments using statistical and geometric modeling. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2011: 203–210

[197] Allan M, Ourselin S, Thompson S, Hawkes DJ, Kelly J, Stoyanov D. Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans Biomed Eng 2013; 60(4): 1050–1058

[198] Reiter A, Allen PK, Zhao T. Appearance learning for 3D tracking of robotic surgical tools. Int J Robot Res 2014; 33(2): 342–356

[199] Reiter A, Goldman RE, Bajo A, Iliopoulos K, Simaan N, Allen PK. A learning algorithm for visual pose estimation of continuum robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2011: 2390–2396

[200] García-Peraza-Herrera LC, Li W, Fidon L, Gruijthuijsen C, Devreker A, Attilakos G, Deprest J, Poorten EV, Stoyanov D, Vercauteren T, Ourselin SE. Toolnet: holistically-nested real-time segmentation of robotic surgical tools. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017: 5717–5722

[201] Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI. Automatic instrument segmentation in robot-assisted surgery using deep learning. IEEE International Conference on Machine Learning and Applications (ICMLA) 2018: 624–628

[202] Shademan A, Decker RS, Opfermann JD, Leonard S, Krieger A, Kim PCW. Supervised autonomous robotic soft tissue surgery. Sci Transl Med 2016; 8(337): 337ra64

[203] Nakamoto M, Ukimura O, Gill IS, Mahadevan A, Miki T, Hashizume M, Sato Y. Realtime organ tracking for endoscopic augmented reality visualization using miniature wireless magnetic tracker. International Workshop on Medical Imaging and Virtual Reality 2008: 359–366

[204] Nosrati MS, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A, Abugharbieh R, Hamarneh G. Efficient multi-organ segmentation in multi-view endoscopic videos using pre-operative priors. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2014: 324–331

[205] Bilodeau GA, Shu Y, Cheriet F. Multistage graph-based segmentation of thoracoscopic images. Comput Med Imaging Graph 2006; 30(8): 437–446

[206] Tjoa MP, Krishnan SM, Kugean C, Wang P, Doraiswaini R. Segmentation of clinical endoscopic image based on homogeneity and hue. International Conference of the IEEE Engineering in Medicine and Biology Society 2001: 2665–2668

[207] Figueiredo IN, Moreno JC, Prasath VBS, Figueiredo PN. A segmentation model and application to endoscopic images. International Conference Image Analysis and Recognition 2012: 164–171

[208] Wu S, Nakao M, Matsuda T. Continuous lung region segmentation from endoscopic images for intra-operative navigation. Comput Biol Med 2017; 87(1): 200–210

[209] Bodenstedt S, Wagner M, Mayer B, Stemmer K, Kenngott H, Müller-Stich B, Dillmann R, Speidel S. Image-based laparoscopic bowel measurement. Int J CARS 2016; 11(3): 407–419

[210] Chhatkuli A, Bartoli A, Malti A, Collins T. Live image parsing in uterine laparoscopy. International Symposium on Biomedical Imaging 2014: 1263–1266

[211] Moccia S, Foti S, Rossi SM, Rota I, Scotti M, Toffoli S, Mattos LS, Momi ED, Frontoni E. FCNN-based segmentation of kidney vessels—towards constraints definition for safe robot-assisted nephrectomy. Joint Workshop on New Technologies for Computer/Robot Assisted Surgery 2018

[212] Rosen J, Hannaford B, Richards CG, Sinanan MN. Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 2001; 48(5): 579–591

[213] Lin HC, Shafran I, Yuh D, Hager GD. Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 2006; 11(5): 220–230

[214] Murali A, Sen S, Kehoe B, Garg A, McFarland S, Patil S, Boyd WD, Lim S, Abbeel P, Goldberg K. Learning by observation for surgical subtasks: multilateral cutting of 3D viscoelastic and 2D orthotropic tissue phantoms. IEEE International Conference on Robotics and Automation (ICRA) 2015: 1202–1209

[215] Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N. Statistical modeling and recognition of surgical workflow. Med Image Anal 2012; 16(3): 632–641

[216] Ahmidi N, Tao L, Sefati S, Gao Y, Lea C, Haro BB, Zappella L, Khudanpur S, Vidal R, Hager GD. A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Trans Biomed Eng 2017; 64(9): 2025–2041

[217] Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R. Sparse hidden Markov models for surgical gesture classification and skill evaluation. International Conference on Information Processing in Computer-Assisted Interventions 2012: 167–177

[218] Zappella L, Béjar B, Hager G, Vidal R. Surgical gesture classification from video and kinematic data. Med Image Anal 2013; 17(7): 732–745

[219] Despinoy F, Bouget D, Forestier G, Penet C, Zemiti N, Poignet P, Jannin P. Unsupervised trajectory segmentation for surgical gesture recognition in robotic training. IEEE Trans Biomed Eng 2016; 63(6): 1280–1291

[220] Murali A, Garg A, Krishnan S, Pokorny FT, Abbeel P, Darrell T, Goldberg K. TSC-DL: unsupervised trajectory segmentation of multi-modal surgical demonstrations with deep learning. IEEE International Conference on Robotics and Automation (ICRA) 2016: 4150–4157

[221] Krishnan S, Garg A, Patil S, Lea C, Hager G, Abbeel P, Goldberg K. Transition state clustering: unsupervised surgical trajectory segmentation for robot learning. Int J Robot Res 2017; 36(13–14): 1595–1618

[222] Kang H, Wen JT. Endobot: a robotic assistant in minimally invasive surgeries. IEEE Int Conf Robot Autom 2001; 2: 2031–2036 (ICRA)

[223] Nageotte F, Zanne P, Doignon C, de Mathelin M. Stitching planning in laparoscopic surgery: towards robot-assisted suturing. Int J Robot Res 2009; 28(10): 1303–1321

[224] Jackson RC, Çavuşoğlu MC. Needle path planning for autonomous robotic surgical suturing. IEEE International Conference on Robotics and Automation (ICRA) 2013: 1669–1675

[225] Sen S, Garg A, Gealy DV, McKinley S, Jen Y, Goldberg K. Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization. IEEE International Conference on Robotics and Automation (ICRA) 2016: 4178–4185

[226] Staub C, Osa T, Knoll A, Bauernschmitt R. Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery. IEEE International Conference on Robotics and Automation (ICRA) 2010: 4585–4590

[227] Leonard S, Wu KL, Kim Y, Krieger A, Kim PCW. Smart tissue anastomosis robot (STAR): a vision-guided robotics system for laparoscopic suturing. IEEE Trans Biomed Eng 2014; 61(4): 1305–1317

[228] Zhong F, Wang Y, Wang Z, Liu YH. Dual-arm robotic needle insertion with active tissue deformation for autonomous suturing. IEEE Robot Autom Lett 2019; 4(3): 2669–2676

[229] Mayer H, Gomez F, Wierstra D, Nagy I, Knoll A, Schmidhuber J. A system for robotic heart surgery that learns to tie knots using recurrent neural networks. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2006: 543–548

[230] Mayer H, Nagy I, Burschka D, Knoll A, Braun EU, Lange R, Bauernschmitt R. Automation of manual tasks for minimally invasive surgery. International Conference on Autonomic and Autonomous Systems (ICAS) 2008: 260–265

[231] Jvd B. Miller S, Duckworth D, Hu H, Wan A, Fu X-Y, Goldberg K, Abbeel P. Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. IEEE International Conference on Robotics and Automation (ICRA) 2010: 2074–2081

[232] Schulman J, Gupta A, Venkatesan S, Tayson-Frederick M, Abbeel P. A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013: 4111–4117

[233] Kehoe B, Kahn G, Mahler J, Kim J, Lee A, Lee A, Nakagawa K, Patil S, Boyd WD, Abbeel P, Goldberg K. Autonomous multilateral debridement with the raven surgical robot. IEEE International Conference on Robotics and Automation (ICRA) 2014: 1432–1439

[234] Le HND, Opfermann JD, Kam M, Raghunathan S, Saeidi H, Leonard S, Kang JU, Krieger A. Semi-autonomous laparoscopic robotic electro-surgery with a novel 3D Endoscope. IEEE International Conference on Robotics and Automation (ICRA) 2018: 6637–6644

[235] Nichols KA, Okamura AM. Autonomous robotic palpation: machine learning techniques to identify hard inclusions in soft tissues. IEEE International Conference on Robotics and Automation (ICRA) 2013: 4384–4389

[236] Nagy DÁ, Nagy TD, Elek R, Rudas IJ, Haidegger T. Ontology-based surgical subtask automation, automating blunt dissection. J Med Robot Res 2018; 3(03n04): 1841005

[237] Nagy TD, Haidegger T. A DVRK-based framework for surgical subtask automation. Acta Polytech Hung 2019; 16(8): 61–78

[238] Awtar S, Trutna TT, Nielsen JM, Abani R, Geiger J. FlexDex™: a minimally invasive surgical tool with enhanced dexterity and intuitive control. J Med Device 2010; 4(3): 035003–035011

[239] Miyazaki R, Hirose K, Ishikawa Y, Kanno T, Kawashima K. A master-slave integrated surgical robot with active motion transformation using wrist axis. IEEE/ASME Trans Mechatron 2018; 23(3): 1215–1225

[240] Roche ET, Horvath MA, Wamala I, Alazmani A, Song S-E, Whyte W, Machaidze Z, Payne CJ, Weaver JC, Fishbein G, Kuebler J, Vasilyev NV, Mooney DJ, Pigula FA, Walsh CJ. Soft robotic sleeve supports heart function. Sci Transl Med 2017; 9(373): eaaf3925

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Nos. 51722507, 51435010 and 91648103), and in part by the National Key R&D Program of China (No. 2017YFC0110800).

Compliance with ethics guidelines

Yuyang Chen, Shu’an Zhang, Zhonghao Wu, Bo Yang, Qingquan Luo, and Kai Xu declare that they have no financial conflicts of interest. This manuscript is a review article and does not involve a research protocol requiring approval by a relevant institutional review board or ethics committee.

RIGHTS & PERMISSIONS

2020 Higher Education Press