Interaction model estimation-based robotic force-position coordinated optimization for rigid-soft heterogeneous contact tasks

Abstract

Inspired by Model Predictive Interaction Control (MPIC), this paper proposes differential models for estimating contact geometric parameters and normal-friction forces and formulates an optimal control problem with multiple constraints to allow robots to perform rigid-soft heterogeneous contact tasks. Within the MPIC, robot dynamics are linearized, and Extended Kalman Filters are used for the online estimation of geometry-aware parameters. Meanwhile, a geometry-aware Hertz contact model is introduced for the online estimation of contact forces. We then implement the force-position coordinate optimization by incorporating the contact parameters and interaction force constraints into a gradient-based optimization MPC. Experimental validations were designed for two contact modes: “single-point contact” and “continuous contact”, involving materials with four different Young’s moduli and tested in human arm “relaxation-contraction” task. Results indicate that our framework ensures consistent geometry-aware parameter estimation and maintains reliable force interaction to guarantee safety. Our method reduces the maximum impact force by 50% and decreases the average force error by 42%. The proposed framework has potential applications in medical and industrial tasks involving the manipulation of rigid, soft, and deformable objects.

Keywords

Heterogeneous contact / Interaction model estimation / Coordination optimization / Model Predictive Control

1 CRediT authorship contribution statement

Haochen Zheng: Writing - original draft. Xueqian Zhai: Software. Hongmin Wu: Methodology. Jia Pan: Resources. Zhihao Xu: Conceptualization. Xuefeng Zhou: Supervision.

2 Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

3 Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (62203126), the Science and Technology Plan Project of Guangdong Province (2023A0505010014), the Innovation and Technology Commission of the HKSAR Government under the InnoHK Initiative, and in part by the Key Areas R&D Program of Dongguan City (20201200300062), and the GDAS’ Project of Science and Technology Development (2022GDASZH-2022010108).

References

[1] Mohammad Mir, Jiawen Chen, Aneri Patel, Meghan R. Pinezich, Brandon A. Guenthart, Gordana Vunjak-Novakovic, Jinho Kim,A minimally invasive robotic tissue palpation device, IEEE Trans. Biomed. Eng. Eng. (2024).

[2] Qinghua Huang, Jiakang Zhou, ZhiJun Li, Review of robot-assisted med-ical ultrasound imaging systems: Technology and clinical applications, Neurocomputing 559 (2023) 126790.

[3] Yingxuan Zhang, Feng Ju, Xiaoyong Wei, Dan Wang, Yaoyao Wang, A piezoelectric tactile sensor for tissue stiffness detection with arbitrary contact angle, Sensors 20 (22) (2020) 6607.

[4] Haochen Zheng, Xueqian Zhai, Shichao Gu, Zhaoyang Liao, Jia Pan, Zhi-hao Xu, Xuefeng Zhou, Shixin Mao, Hongmin Wu, Vision-based defects detection and interactive impedance control in robotic automated layup tasks, in: 2023 IEEE International Conference on Robotics and Biomimetics, ROBIO, 2023, pp. 1-6.

[5] Matthias Mayr, Faseeh Ahmad, Konstantinos Chatzilygeroudis, Luigi Nardi, Volker Krueger, Skill-based multi-objective reinforcement learning of in-dustrial robot tasks with planning and knowledge integration, in: 2022 IEEE International Conference on Robotics and Biomimetics, ROBIO, IEEE, 2022, pp. 1995-2002.

[6] Cristian Camilo Beltran-Hernandez, Damien Petit, Ixchel Georgina Ramirez-Alpizar, Takayuki Nishi, Shinichi Kikuchi, Takamitsu Matsubara, Kensuke Harada, Learning force control for contact-rich manipulation tasks with rigid position-controlled robots, IEEE Robot. Autom. Lett. 5 (4) (2020) 5709-5716.

[7] Chao Zeng, Chenguang Yang, Zhehao Jin, Jianwei Zhang, Hierarchical impedance, force, and manipulability control for robot learning of skills, IEEE/ASME Trans. Mechatronics (2024).

[8] Sotaro Katayama, Toshiyuki Ohtsuka, Whole-body model predictive control with rigid contacts via online switching time optimization, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2022, pp. 8858-8865.

[9] Kaidong Liu, Bin Xie, Zhouyang Chen, Zhenhao Luo, Shan Jiang, Zhen Gao, Human-robot skill transferring and inverse velocity admittance control for soft tissue cutting tasks, Agriculture 14 (3) (2024) 394.

[10] Jing Luo, Chaoyi Zhang, Weiyong Si, Yiming Jiang, Chenguang Yang, Chao Zeng, A physical human-robot interaction framework for trajectory adaptation based on human motion prediction and adaptive impedance control, IEEE Trans. Autom. Sci. Eng. (2024).

[11] Basem Adel Aly, Tobias Low, Derek Long, Peter Brett, Craig Baillie, Tactile sensing for tissue discrimination in robotic meat cutting: A feasibility study, J. Food Eng. 363 (2024) 111754.

[12] Tobias Gold, Andreas Völz, Knut Graichen, Model predictive interaction control for robotic manipulation tasks, IEEE Trans. Robot. 39 (1) (2023) 76-89.

[13] Tobias Gold, Andreas Völz, Knut Graichen, Model predictive position and force trajectory tracking control for robot-environment interaction, in: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2020, pp. 7397-7402.

[14] Junjie Shen, Dennis Hong, Optimal linearization via quadratic programming, IEEE Robot. Autom. Lett. 5 (3) (2020) 4572-4579.

[15] Zhihong Jiang, Weigang Zhou, Hui Li, Yang Mo, Wencheng Ni, Qiang Huang, A new kind of accurate calibration method for robotic kinematic parameters based on the extended Kalman and particle filter algorithm, IEEE Trans. Ind. Electron. 65 (4) (2017) 3337-3345.

[16] Tobias Gold, Andreas Völz, Knut Graichen, Model predictive position and force trajectory tracking control for robot-environment interaction, in: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2020, pp. 7397-7402.

[17] Mohamed Elobaid, Giulio Romualdi, Gabriele Nava, Lorenzo Rapetti, Hosameldin Awadalla Omer Mohamed, Daniele Pucci, Online non-linear centroidal MPC for humanoid robots payload carrying with contact-stable force parametrization, in: 2023 IEEE International Conference on Robotics and Automation, ICRA, 2023, pp. 12233-12239.

[18] Guillaume Bellegarda, Katie Byl, An online training method for augmenting MPC with deep reinforcement learning, in: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2020, pp. 5453-5459.

[19] Yuquan Wang, Niels Dehio, Abderrahmane Kheddar, On inverse inertia matrix and contact-force model for robotic manipulators at normal impacts, IEEE Robot. Autom. Lett. 7 (2) (2022) 3648-3655.

[20] Fugui Xie, Zenghui Chong, Xin-Jun Liu, Huichan Zhao, Jinsong Wang, Precise and smooth contact force control for a hybrid mobile robot used in polishing, Robot. Comput.-Integr. Manuf. 83 (2023) 102573.

[21] Alp Aydinoglu, Victor M. Preciado, Michael Posa, Contact-aware controller design for complementarity systems, in: 2020 IEEE International Conference on Robotics and Automation, ICRA, 2020, pp. 1525-1531.

[22] Mohsen Sombolestan, Quan Nguyen, Hierarchical adaptive control for collaborative manipulation of a rigid object by quadrupedal robots, in: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2023, pp. 2752-2759.

[23] Sébastien Kleff, Justin Carpentier, Nicolas Mansard, Ludovic Righetti, On the derivation of the contact dynamics in arbitrary frames: Application to polishing with talos,in: 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), 2022, pp. 512-517.

[24] Zhehao Jin, Dongdong Qin, Andong Liu, Wen-an Zhang, Li Yu, Model predictive variable impedance control of manipulators for adaptive precision-compliance tradeoff, IEEE/ASME Trans. Mechatronics 28 (2) (2023) 1174-1186.

[25] Miroslav Bogdanovic, Majid Khadiv, Ludovic Righetti, Learning variable impedance control for contact sensitive tasks, IEEE Robot. Autom. Lett 5 (4) (2020) 6129-6136.

[26] Zhu Dachang, Du Baolin, Zhu Puchen, Chen Shouyan, Constant force PID control for robotic manipulator based on fuzzy neural network algorithm, Complexity 2020 (1) (2020) 3491845.

[27] Saif Sidhik, Mohan Sridharan, Dirk Ruiken, Towards an online framework for changing-contact robot manipulation tasks, in: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2021, pp. 5203-5210.

[28] Ning Wang, Chuize Chen, Alessandro Di Nuovo, A framework of hybrid force/motion skills learning for robots, IEEE Trans. Cogn. Dev. Syst. 13 (1) (2021) 162-170.

[29] Huanyu Tian, Zhe Han, Yang Wang, Xiaolong Zhu, Weijun Zhang, Zhengjie Wang, Changsheng Li, Xingguang Duan, Virtual-fixture based osteotomy shared control: A framework for human-robot shared surgical osteotomy manipulation, IEEE Trans. Med. Robotics Bionics (2023).

[30] Emanuele Magrini, Alessandro De Luca, Hybrid force/velocity con-trol for physical human-robot collaboration tasks, in: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2016, pp. 857-863.

[31] Junling Fu, Ilaria Burzo, Elisa Iovene, Jianzhuang Zhao, Giancarlo Ferrigno, Elena De Momi, Optimization-based variable impedance control of robotic manipulator for medical contact tasks, IEEE Trans. Instrum. Meas. 73 (2024) 1-8.

[32] Yanan Li, Gowrishankar Ganesh, Nathanaël Jarrassé, Sami Haddadin, Alin Albu-Schaeffer, Etienne Burdet, Force, impedance, and trajectory learning for contact tooling and haptic identification, IEEE Trans. Robot. 34 (5)(2018) 1170-1182.

[33] Yang Li, Wenjie Lin, Zhehao Jin, Xiang Wu, Qun Lu, Hui Dong, Sensor-free variable impedance control using quadratic programming and extended state observer, in: 2024 IEEE International Conference on Industrial Technology, ICIT, 2024, pp. 1-5.

[34] Xing Liu, Panfeng Huang, Zhengxiong Liu, A novel contact state estimation method for robot manipulation skill learning via environment dynamics and constraints modeling, IEEE Trans. Autom. Sci. Eng. 19 (4) (2022) 3903-3913.

[35] Rui Wu, Aude Billard, Learning from demonstration and interactive control of variable-impedance to cut soft tissues, IEEE/ASME Trans. Mechatronics 27 (5) (2022) 2740-2751.

[36] Haohui Huang, Chenguang Yang, C.L. Philip Chen, Optimal robot-environment interaction under broad fuzzy neural adaptive control, IEEE Trans. Cybern. 51 (7) (2021) 3824-3835.

[37] Mohammad Hossein Hamedani, Hamid Sadeghian, Maryam Zekri, Farid Sheikholeslam, Mehdi Keshmiri, Intelligent impedance control using wavelet neural network for dynamic contact force tracking in unknown varying environments, Control Eng. Pract. 113 (2021) 104840.

[38] Tao Pang, Russ Tedrake, A convex quasistatic time-stepping scheme for rigid multibody systems with contact and friction, in: 2021 IEEE International Conference on Robotics and Automation, ICRA, 2021, pp. 6614-6620.

[39] Joonhee Jo, Gyunghoon Park, Yonghwan Oh, Robust walking stabiliza-tion strategy of humanoid robots on uneven terrain via QP-based impedance/admittance control, Robot. Auton. Syst. 154 (2022) 104148.

[40] Jinda Cui, Jiawei Xu, David Saldana, Jeff Trinkle, Toward fine contact inter-actions: Learning to control normal contact force with limited information, in: 2023 IEEE International Conference on Robotics and Automation, ICRA, 2023, pp. 3926-3932.

[41] Jian Li, Yisheng Guan, Haowen Chen, Bing Wang, Tao Zhang, Robotic polishing of unknown-model workpieces with constant normal contact force control, IEEE/ASME Trans. Mechatronics 28 (2) (2023) 1093-1103.

[42] Ross Hartley, Maani Ghaffari, Ryan M. Eustice, Jessy W Grizzle, Contact-aided invariant extended Kalman filtering for robot state estimation, Int. J. Robot. Res. 39 (4) (2020) 402-430.

[43] Jinjun Duan, Yahui Gan, Ming Chen, Xianzhong Dai, Adaptive vari-able impedance control for dynamic contact force tracking in uncertain environment, Robot. Auton. Syst. 102 (2018) 54-65.

[44] Francisco Rubio, Francisco Valero, Carlos Llopis-Albert, A review of mobile robots: Concepts, methods, theoretical framework, and applications, Int. J. Adv. Robot. Syst. 16 (2) (2019) 1729881419839596.

[45] Said Zahmoun, Ahmed Ouammi, Roberto Sacile, Rachid Benchrifa, Enrico Zero, Optimal operation scheduling of a combined wind-hydro system for peak load shaving, IEEE Trans. Autom. Sci. Eng. (2023) 1-14.

[46] Giovanni Faoro, Sabina Maglio, Stefano Pane, Veronica Iacovacci, Arianna Menciassi, An artificial intelligence-aided robotic platform for ultrasound-guided transcarotid revascularization, IEEE Robot. Autom. Lett. 8 (4) (2023) 2349-2356.

[47] Davide Doria, Simone Fani, Andrea Giannini, Tommaso Simoncini, Matteo Bianchi, Enhancing the localization of uterine leiomyomas through cuta-neous softness rendering for robot-assisted surgical palpation applications, IEEE Trans. Haptics 14 (3) (2021) 503-512.