A vision-based humanoid compliant skill transfer framework: Application to robotic cutting tasks

Abstract

Autonomously completing a contact-rich task for multiple manipulation objects remains a challenging problem for robots. To achieve this goal, learning from demonstration has emerged as an efficient method for transferring human-like skills to robots. Existing works primarily focus on trajectory or impedance learning to design force-impedance controllers for specific tasks, which require precise force sensing. However, visual perception plays a critical role in enabling humans to perform dexterous manipulation. To bridge the gap between vision and learning in the control loop, this work proposes a vision-based humanoid compliant skill transfer (VHCST) framework. Considering the lack of vision-impedance mapping, a hybrid tree is introduced as a planning bridge to encode skill parameters across multiple objects. To simplify skill transfer, an observation-wearable demonstration method is employed to capture the position and stiffness of human’s arm. The decoupled learning model incorporates the geometric properties of stiffness ellipsoids, which reside on Riemannian manifolds. Finally, the proposed approach is validated through robotic cutting experiments involving multiple objects. Comparative experimental results demonstrate the effectiveness of the proposed framework.

Keywords

Human–robot skill transfer / Compliant control / Skill generalization / Robotic cutting

CRediT authorship contribution statement

Zhaohong Mai: Writing – original draft, Software, Methodology. Chao Zeng: Writing – review & editing, Supervision, Funding acquisition. Ning Wang: Validation, Formal analysis. Chenguang Yang: Resources, Project administration.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the UKRI Guarantee funding for Horizon Europe MSCA Postdoctoral Fellowships (EP/Z00117X/1).

Appendix A. Supplementary data

Supplementary material related to this article can be found online at https://doi.org/10.1016/j.birob.2026.100280.

References

[1] Botao Lin, Shuang Song, Jiaole Wang, Variable stiffness methods of flexible robots for minimally invasive surgery: A review, Biomim. Intell. Robot., (2024), 100168.

[2] Moses C. Nah, Johannes Lachner, Neville Hogan, Robot control based on motor primitives: A comparison of two approaches, Int. J. Robot. Res. 43 (12) (2024) 1959-1991.

[3] Sami Haddadin, Erfan Shahriari, Unified force-impedance control, Int. J. Robot. Res. 43 (13) (2024) 2112-2141.

[4] Vincenzo Lippiello, Giuseppe Andrea Fontanelli, Fabio Ruggiero, Image-based visual-impedance control of a dual-arm aerial manipulator, IEEE Robot. Autom. Lett. 3 (3) (2018) 1856-1863.

[5] Chenguang Yang, et al., Human-like adaptation of force and impedance in stable and unstable interactions, IEEE Trans. Robot. 27 (5) (2011) 918-930.

[6] Markku Suomalainen, Yiannis Karayiannidis, Ville Kyrki, A survey of robot manipulation in contact, Robot. Auton. Syst. 156 (2022) 104224.

[7] Harish Ravichandar, et al., Recent advances in robot learning from demonstration, Annu. Rev. Control. Robot. Auton. Syst. 3 (1) (2020) 297-330.

[8] Chao Zeng, et al., Hierarchical impedance, force, and manipulability control for robot learning of skills, IEEE/ASME Trans. Mechatronics, (2024).

[9] Zhiwei Liao, et al., Simultaneously learning of motion, stiffness, and force from human demonstration based on riemannian dmp and qp optimization, IEEE Trans. Autom. Sci. Eng., (2024).

[10] Chao Zeng, et al., An approach for robotic leaning inspired by biomimetic adaptive control, IEEE Trans. Ind. Inform. 18 (3) (2021) 1479-1488.

[11] Sylvain Calinon, A tutorial on task-parameterized movement learning and retrieval, Intell. Serv. Robot. 9 (2016) 1-29.

[12] Chenzui Li, et al., Towards Robo-Coach: Robot interactive stiffness/position adaptation for human strength and conditioning training, 2024 IEEE International Conference on Robotics and Automation, ICRA, IEEE, 2024, pp. 860-866.

[13] Martijn J.A. Zeestraten, et al., An approach for imitation learning on Riemannian manifolds, IEEE Robot. Autom. Lett. 2 (3) (2017) 1240-1247.

[14] Sylvain Calinon, Gaussians on Riemannian manifolds: Applications for robot learning and adaptive control, IEEE Robot. Autom. Mag. 27 (2) (2020) 33-45.

[15] Fares J. Abu-Dakka, Matteo Saveriano, Ville Kyrki, A unified formulation of geometry-aware discrete dynamic movement primitives, Neurocomputing 598 (2024) 128056.

[16] Rui Wu, He Zhang, Jie Zhao, Robot variable impedance skill transfer and learning framework based on a simplified human arm impedance model, IEEE Access 8 (2020) 225627-225638.

[17] Yuqiang Wu, et al., A framework for autonomous impedance regulation of robots based on imitation learning and optimal control, IEEE Robot. Autom. Lett. 6 (1) (2020) 127-134.

[18] Noémie Jaquier, Sylvain Calinon, Gaussian mixture regression on symmetric positive definite matrices manifolds: Application to wrist motion estimation with sEMG, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, IEEE, 2017, pp. 59-64.

[19] Luca Beber, et al., A passive variable impedance control strategy with viscoelastic parameters estimation of soft tissues for safe ultrasonography, 2024 IEEE International Conference on Robotics and Automation, ICRA, IEEE, 2024, pp. 1298-1304.

[20] Haochen Zheng, et al., Interaction model estimation-based robotic force-position coordinated optimization for rigid–soft heterogeneous contact tasks, Biomim. Intell. Robot. 5 (1) (2025) 100194.

[21] Matteo Iovino, et al., A survey of behavior trees in robotics and ai, Robot. Auton. Syst. 154 (2022) 104096.

[22] Jacques Cloete, Wolfgang Merkt, Ioannis Havoutis, Adaptive manipulation using behavior trees, 2024, arXiv preprint arXiv:2406.14634.

[23] Simona Gugliermo, et al., Learning behavior trees from planning experts using decision tree and logic factorization, IEEE Robot. Autom. Lett. 8 (6) (2023) 3534-3541.

[24] Kevin Zhang, et al., Leveraging multimodal haptic sensory data for robust cutting, 2019 IEEE-RAS 19th International Conference on Humanoid Robots, Humanoids, IEEE, 2019, pp. 409-416.

[25] Prajjwal Jamdagni, Yan-Bin Jia, Robotic cutting of fruits and vegetables: Modeling the effects of deformation, fracture toughness, knife edge geometry, and motion, IEEE Trans. Robot., (2024).

[26] Ryan Wright, et al., Safely and autonomously cutting meat with a collaborative robot arm, Sci. Rep. 14 (1) (2024) 299.

[27] Chenguang Yang, et al., Interface design of a physical human–robot interaction system for human impedance adaptive skill transfer, IEEE Trans. Autom. Sci. Eng. 15 (1) (2017) 329-340.

[28] Zhenjia Xu, et al., RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects, in: Proceedings of Robotics: Science and Systems, RSS, 2023.

[29] Mathew Jose Pollayil, et al., Choosing stiffness and damping for optimal impedance planning, IEEE Trans. Robot. 39 (2) (2022) 1281-1300.

[30] Rajendra Bhatia, Positive Definite Matrices, Princeton University Press, (2009).

[31] Xavier Pennec, Pierre Fillard, Nicholas Ayache, A Riemannian framework for tensor computing, Int. J. Comput. Vis. 66 (2006) 41-66.

[32] Liwen Situ, et al., Human multi-dimensional stiffness skills transfer for robot teleoperation system, 2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC, IEEE, 2024, pp. 321-327.

[33] Glenn Jocher, Ayush Chaurasia, Jing Qiu, Ultralytics YOLOv8, (2023),URL: https://github.com/ultralytics/ultralytics.

[34] Juan R. Terven, Diana M. Córdova-Esparza, Kin2. a kinect 2 toolbox for MATLAB, Sci. Comput. Program. 130 (2016) 97-106.