Haptic and visual augmented reality interface for programming welding robots

Abstract

It is a challenging task for operators to program a remote robot for welding manipulation depending only on the visual information from the remote site. This paper proposes an intuitive user interface for programming welding robots remotely using augmented reality (AR) with haptic feedback. The proposed system uses a depth camera to reconstruct the surfaces of workpieces. A haptic input device is used to allow users to define welding paths along these surfaces. An AR user interface is developed to allow users to visualize and adjust the orientation of the welding torch. Compared with the traditional robotic welding path programming methods which rely on prior CAD models or contact between the robot end-effector and the workpiece, this proposed approach allows for fast and intuitive remote robotic welding path programming without prior knowledge of CAD models of the workpieces. The experimental results show that the proposed approach is a user-friendly interface and can assist users in obtaining an accurate welding path.

Keywords

Robot programming / Human-robot interaction / Robotic welding / Haptic feedback / Augmented reality (AR)

References

[1] Rentzos L, Vourtsis C, Mavrikios D et al (2014) Using VR for complex product design. In: Proceedings of the international conference on virtual, augmented and mixed reality 2014, Crete, Greece, 22–27 June, pp 455–464

[2] Mavrikios D, Karabatsou V, Fragos D, et al. A prototype virtual reality based demonstrator for immersive and interactive simulation of welding processes. Int J Comput Integr Manufact, 2006, 19(3): 294-300.

[3] Makris S, Karagiannis P, Koukas S, et al. S. Augmented reality system for operator support in human-robot collaborative assembly. CIRP Ann Manufact Technol, 2016, 65(1): 61-64.

[4] Kon T, Oikawa T, Choi Y et al (2006) A method for supporting robot’s actions using virtual reality in a smart space. In: Proceedings of the SICE-ICASE international joint conference 2006, Busan, Korea, 18–21 Oct, pp 3519–3522

[5] Andersson N, Argyrou A, Nägele F, et al. AR-enhanced human-robot-interaction: methodologies, algorithms tools. Procedia CIRP, 2016, 44: 193-198.

[6] Chotiprayanakul P, Wang D, Kwok N et al (2008) A haptic base human robot interaction approach for robotic grit blasting. In: Proceedings of the 25th international symposium on automation and robotics in construction, Vilnius, Lithuania, 26–29 June 2008, pp 148–154

[7] El Saddik A. The potential of haptics technologies. IEEE Instrum Meas Mag, 2007, 10(1): 10-17.

[8] Velanas SV, Tzafestas CS (2010) Human telehaptic perception of stiffness using an adaptive impedance reflection bilateral teleoperation control scheme. In: Proceedings of the IEEE 19th international symposium in robot and human interactive communication, Viareggio, Italy, 13–16 Sept, pp 21–26

[9] Rosenberg LB (1993) Virtual fixtures: perceptual tools for telerobotic manipulation. In: Proceedings of the IEEE virtual reality annual international symposium, Seattle, USA, 18–22 Sept, pp 76–82

[10] Li M, Kapoor A, Taylor RH. Telerobotic control by virtual fixtures for surgical applications. Springer Tracts Adv Robot, 2007, 31(1): 381-401.

[11] Bolopion A, Régnier S. A review of haptic feedback teleoperation systems for micromanipulation and microassembly. IEEE Trans Autom Sci Eng, 2013, 10(3): 496-502.

[12] Xia, T, Leonard S, Kandaswamy I et al (2013) Model-based telerobotic control with virtual fixtures for satellite servicing tasks. In: Proceedings of the IEEE international conference on robotics and automation (ICRA) 2013, Karlsruhe, Germany, 6–10 May, pp 1479–1484

[13] Aleotti J, Reggiani M. Evaluation of virtual fixtures for a robot programming by demonstration interface. IEEE Trans Syst Man Cybern Part A: Syst Hum, 2005, 35(4): 536-545.

[14] Wang Y, Chen Y, Nan Z et al (2006) Study on welder training by means of haptic guidance and virtual reality for arc welding. In: Proceedings of the IEEE international conference on robotics and biomimetics 2006, Kunming, China, 17–20 Dec, pp 954–958

[15] Nichol CI, Manic M (2009) Video game device haptic interface for robotic arc welding. In: Proceedings of the 2nd conference on human system interactions 2009, Catania, Italy, 21–23 May, pp 648–653

[16] Reddy PA, Reddy TD. Design and development of a telemanipulated welding robot with visual and haptic feedback. Int J Res Eng Technol, 2016, 5(8): 371-376.

[17] Smits R (2016). KDL: kinematics and dynamics library. Retrieved 9 Dec 2016, http://www.orocos.org/kdl

[18] Leeper A, Chan S, Salisbur K (2012) Point clouds can be represented as implicit surfaces for constraint-based haptic rendering. In: IEEE international conference on robotics and automation 2012, St Paul, USA, 14–18 May, pp 5000–5005

[19] Garrido-Jurado S, Muñoz-Salinas R, Marín-Jiménez MJ. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recog, 2014, 47(6): 2280-2292.

Funding

ASTAR (Thematic Program on Industrial Robotics, Project No. 1225100001)

China Scholarship Council http://dx.doi.org/10.13039/501100004543