Human-robot collaborative handling of curtain walls using dynamic motion primitives and real-time human intention recognition

Abstract

Human-robot collaboration fully leverages the strengths of both humans and robots, which is crucial for handling large, heavy objects at construction sites. To address the challenges of human-machine cooperation in handling large-scale, heavy objects — specifically building curtain walls — a human-robot collaboration system was designed based on the concept of “human-centered with machine support”. This system allows the handling of curtain walls according to different human intentions. First, a robot trajectory learning and generalization model based on dynamic motion primitives was developed. The operator’s motion intent was then characterized by their speed, force, and torque, with the force impulse introduced to define the operator’s intentions for acceleration and deceleration. Finally, a collaborative experiment was conducted on an experimental platform to validate the robot’s understanding of human handling intentions and to verify its ability to handle curtain wall. Collaboration between humans and robots ensured a smooth and labor-saving handling process.

Keywords

Robots / Dynamic motion primitives / Human-machine collaboration / Curtain wall handling

CRediT authorship contribution statement

Fengming Li: Methodology, Funding acquisition, Conceptualization. Huayan Sun: Data curation. Enguang Liu: Project administration, Methodology, Conceptualization. Fuxin Du: Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by 2022 Doctoral Fund Project of Shandong Jianzhu University (X22012Z) and the National Natural Science Foundation of China (U20A20283).

References

[1] S.N. Yu, S.Y. Lee, C.S. Han, K.Y. Lee, S.H. Lee, Development of the curtain wall installation robot: Performance and efficiency tests at a construction site, Auton. Robots 22 (2007) 281-291.

[2] H. Wu, W. Yan, Z. Xu, T. Cheng, A framework of robot skill learning from complex and long-horizon tasks, IEEE Trans. Autom. Sci. Eng. 19 (4) (2021) 3628-3638.

[3] O. Kroemer, S. Niekum, G. Konidaris, A review of robot learning for manipulation: Challenges, representations, and algorithms, J. Mach. Learn. Res. 22 (1) (2021) 1395-1476.

[4] J. Holas, I. Farkaš, Adaptive skill acquisition in hierarchical reinforce-ment learning, in: International Conference on Artificial Neural Networks, Springer International Publishing, Cham, 2020, pp. 383-394.

[5] J.M.D. Delgado, L. Oyedele, Robotics in construction: A critical review of the reinforcement learning and imitation learning paradigms, Adv. Eng. Inform. 54 (2022) 101787.

[6] Y. Hu, M. Cui, J. Duan, W. Liu, D. Huang, A. Knoll, G. Chen, Model predictive optimization for imitation learning from demonstrations, Robot. Auton. Syst. 163 (2023) 104381.

[7] A. Prados, S. Garrido, R. Barber, Learning and generalization of task-parameterized skills through few human demonstrations, Eng. Appl. Artif. Intell. 133 (2024) 108310.

[8] F. Iodice, Y. Wu, W. Kim, F. Zhu, D.M. Elena, A. Ajoudani, Learning co-operative dynamic manipulation skills from human demonstration videos, Mechatronics 85 (2022) 102807.

[9] Y. Shen, J. Wang, C. Feng, Q. Wang, Hybrid-driven autonomous excavator trajectory generation combining empirical driver skills and optimization, Autom. Constr. 165 (2024) 105523.

[10] Y. Shen, J. Wang, C. Feng, Y. Liu, D. Li, Y. Song, X. Qu, Jointly estimating the most likely driving paths and destination locations with incomplete vehicular trajectory data, Transp. Res. C 155 (2023) 104283.

[11] C. Liu, G. Peng, Y. Xia, J. Li, C. Yang, Robot skill learning system of multi-space fusion based on dynamic movement primitives and adaptive neural network control, Neurocomputing 574 (2024) 127248.

[12] W. Li, Y. Wang, Y. Liang, D.T. Pham, Learning from demonstration for au-tonomous generation of robotic trajectory: Status quo and forward-looking overview, Adv. Eng. Inform. 62 (2024) 102625.

[13] H. Zhao, Y. Chen, X. Li, H. Ding, Robotic peg-in-hole assembly based on reversible dynamic movement primitives and trajectory optimization, Mechatronics 95 (2023) 103054.

[14] L. Han, H. Yuan, W. Xu, Y. Huang, Modified dynamic movement prim-itives: robot trajectory planning and force control under curved surface constraints, IEEE Trans. Cybern. (2022).

[15] F. Yang, Q. Zheng, L. Chen, X. Tan, P. Che, Research on human behavior intention perception method based on wearable sensors, IEEE Access (2024).

[16] H.I. Lin, C.S.G. Lee, Speed-accuracy optimization for skill learning, in: 2009 IEEE International Conference on Robotics and Automation, IEEE, 2009, pp. 2506-2511.

[17] Q. Yang, C. Xie, R. Tang, H. Liu, R. Song, Hybrid active control with human intention detection of an upper-limb cable-driven rehabilitation robot, IEEE Access 8 (2020) 195206-195215.

[18] Y. Zhang, K. Ding, J. Hui, J. Lv, X. Zhou, P. Zheng, Human-object in-tegrated assembly intention recognition for context-aware human-robot collaborative assembly, Adv. Eng. Inform. 54 (2022) 101792.

[19] Y. Pan, C. Chen, Z. Zhao, T. Hu, J. Zhang, Robot teaching system based on hand-robot contact state detection and motion intention recognition, Robot. Comput. -Integr. Manuf. 81 (2023) 102492.

[20] Z. Lu, N. Wang, M. Li, C. Yang, Incremental motor skill learning and gen-eralization from human dynamic reactions based on dynamic movement primitives and fuzzy logic system, IEEE Trans. Fuzzy Syst. 30 (6) (2021) 1506-1515.