Method for Latency Measurement of Visual Feedback-Based Master–Slave Minimally Invasive Surgical Robot

Abstract

To measure the latency between human motion stimulation and stereo image display response in a visual feedback-based minimally invasive surgical (MIS) robotic system, a method was proposed by comparing the orientations of input and output events through image-processing technology. This method used a black bar to keep pace with the measured joint rotating at a number of speeds. During tests, an external camera was placed in front of the apparatus with a proper visual field, so that it can simultaneously view orientations of both bars fixed on the corresponding joints. After quantitatively analyzing the accuracy of the proposed measurement method, the method was applied to a visual feedback-based master–slave robotic system with two-degrees-of-freedom. Experimental results show that the latency of the overall system was approximately 250 ms, and the opposite clearance of the measured joint was in the range of 1.7°–1.9°.

Keywords

Minimally invasive surgical (MIS) robot / Latency measurement / Real time / Visual feedback

References

[1] Kang SB, Park JS, Kim DW, et al. Intraoperative technical difficulty during laparoscopy-assisted surgery as a prognostic factor for colorectal cancer. Dis Colon Rectum, 2010, 53(10): 1400-1408.

[2] Sang HQ, Wang SX, Li JM, et al. Control design and implementation of a novel master–slave surgery robot system, MicroHand A. Int J Med Robot Comput Assist Surg, 2011, 7(3): 334-347.

[3] Gidaro S, Altobelli E, Falavolti C, et al. Vesicourethral anastomosis using novel telesurgical system with haptic sensation, the Telelap Alf-X: a pilot study. Surg Technol Int, 2014, 24: 35.

[4] Ghodoussi M, Butner SE, Wang YL (2002) Robotic surgery—the transatlantic case. In: Proceedings of IEEE International Conference on Robotics and Automation, Washington, USA, pp 1882–1888

[5] Wang W, Li JM, Wang SX, et al. System design and animal experiment study of a novel minimally invasive surgical robot. Int J Med Robot Comput Assist Surg, 2016, 12: 73-84.

[6] Li JH, Wang XS, Xing Y, et al. Optimization algorithm for operation comfortability of master manipulator of minimally invasive surgery robot. Trans Tianjin Univ, 2016, 22(2): 95-104.

[7] Perez M, Xu S, Chauhan S, et al. Impact of delay on telesurgical performance: study on the robotic simulator dV-Trainer. Int J Comput Assist Radiol Surg, 2016, 11(4): 581-587.

[8] Marescaux J, Leroy J, Gagner M, et al. Transatlantic robot-assisted telesurgery. Nature, 2001, 413(6854): 379-380.

[9] Wu WX, Dong YJ, Hoover A. Measuring digital system latency from sensing to actuation at continuous 1-ms resolution. Presence Teleoper Virtual Environ, 2013, 22(1): 20-35.

[10] Adelstein BD, Johnston ER, Ellis SR. Dynamic response of electromagnetic spatial displacement trackers. Presence, 1996, 5(3): 302-318.

[11] Jansen J, Bulterman DCA (2013) User-centric video delay measurements. In: Proceedings of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, Oslo, Norway, pp 37–42

[12] Kryczka A, Arefin A, Nahrstedt K (2013) AvCloak: A tool for black box latency measurements in video conferencing applications. In: Proceedings of IEEE International Symposium on Multimedia, Anaheim, USA, pp 271–278

[13] Hill R, Madden C, Hengel AVD et al (2009) Measuring latency for video surveillance systems. In: Proceedings of Digital Image Computing: Techniques and Applications, Melbourne, Australia, pp 89–95

[14] Steed A (2008) A simple method for estimating the latency of interactive, real-time graphics simulations. In: Proceedings of ACM Symposium on Virtual Reality Software and Technology, Bordeaux, France, pp 123–129

[15] Drioli C, Foresti GL (2015) Time-varying delay measurement of video capture-to-display components with application to visual servoing. Signal Processing Image Communication 39 (Part A): 84–97

[16] Su MJ, Guan YS, Hu J et al (2013) Development and analysis of a bilateral control system for modular master-slave robots with P-P tracking capability. In: Proceedings of IEEE International Conference on Mechatronics and Automation (ICMA), Takamatsu, Japan, pp 336–341

[17] Shiotsuki T (2006) Time delay compensation of human operator in position tracking tasks. In: Proceedings of SICE-ICASE, Busan, Republic of Korea, pp 1504–1507

[18] Rohde M, van Dam LCJ, Ernst MO. Predictability is necessary for closed-loop visual feedback delay adaptation. J Vision, 2014, 14(3): 4

[19] Friston S, Karlström P, Steed A. The effects of low latency on pointing and steering tasks. IEEE Trans Visual Comput Graphics, 2016, 22(5): 1605-1615.

[20] Xu S, Perez M, Yang K, et al. Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer ^® simulator. Surg Endosc, 2014, 28(9): 2569-2576.

[21] Wang YM. The principle, algorithm, and application of moment functions in image, 2002, 1 China: East China University of Science and Technology Press, Shanghai.

[22] Su H, Li JM, Zhang HF, et al. Using motion parallax for laparoscopic surgery. International Journal of Medical Robotics & Computer Assisted Surgery, 2015, 12(3): 399-409.