Reactive collision-free motion generation for tightly coupled coordinated bimanual manipulation

Abstract

Dual-arm robots operating in dynamic, human-centered environments must be reactive, dexterous, and safe while intelligently coordinating both arms to perform task-space-constrained manipulations. To address this challenge, we propose a general framework comprising three interrelated modules for dual-arm robots, which integrates tightly coupled coordinated bimanual manipulation with reactive collision-free motion: i) The multiple task-priority joint control module achieves tightly coupled bimanual coordination by taking advantage of the cooperative dual task space, including the fixed relative pose of the dual-arm end-effectors, the flexible absolute pose control of the end-effectors, and satisfying joint limits. ii) The nominal trajectory module, based on the vector field that combines the target attraction force and the manipulability force, dynamically generates nominal reference trajectory for the end-effectors. iii) The safety filter module, based on control barrier function, locally reshapes the nominal reference trajectory in real time to generate collision-free reference trajectory. Various planning methods are thoroughly validated in cluttered and dynamic simulation scenarios. The proposed method is able to reliably accomplish constrained tasks while existing solutions perform poorly. In real-world experiments, a 14-degree-of-freedom dual-arm robot transports multiple cups by grasping a tray in an environment with random disturbances. Experimental results demonstrate that the robot prevents the cups from sliding off by adaptively adjusting the tray’s tilt angle. Simultaneously, bimanual reactive motion enables the robot to successfully avoid dynamic obstacles while maintaining task execution.

Graphical abstract

Keywords

redundant manipulator / bimanual manipulation / collision avoidance / kinematics / reactive and sensor-based planning

References

[1] Billard A , Kragic D . Trends and challenges in robot manipulation. Science, 2019, 364(6446): eaat8414

[2] Lavalle S M. Planning Algorithms. Cambridge University Press, 2006

[3] Gafur N, Kanagalingam G, Ruskowski M. Dynamic collision avoidance for multiple robotic manipulators based on a non-cooperative multi-agent game. 2021, arXiv preprint arXiv:2103.00583

[4] Jiang Y , Wang J , Yang M , Zhong H , Zhang H , Yang C . Discrete‐time neural network based mode switching optimal control scheme for mobile dual‐manipulator robots. Journal of Field Robotics, 2024, 42(5): 1665–1678

[5] Mirrazavi Salehian S S , Figueroa N , Billard A . A unified framework for coordinated multi-arm motion planning. International Journal of Robotics Research, 2018, 37(10): 1205–1232

[6] Zimmermann S , Hakimifard G , Zamora M , Poranne R , Coros S . A multi-level optimization framework for simultaneous grasping and motion planning. IEEE Robotics and Automation Letters, 2020, 5(2): 2966–2972

[7] Dastider A, Fang H, Lin M. APEX: Ambidextrous dual-arm robotic manipulation using collision-free generative diffusion models. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Abu Dhabi: IEEE, 2024, 9526–9533

[8] Krebs F , Asfour T . A bimanual manipulation taxonomy. IEEE Robotics and Automation Letters, 2022, 7(4): 11031–11038

[9] Rigatos G, Busawon K, Abbaszadeh M, Pomares J, Gao Z, Zouari F. Flatness-based control in successive loops for dual-arm robotic manipulators. In: IEEE Conference on Control Technology and Applications (CCTA). Newcastle upon Tyne: IEEE, 2024, 793–798

[10] Adorno B V, Fraisse P, Druon S. Dual position control strategies using the cooperative dual task-space framework. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2010, 3955–3960

[11] Ames A D , Xu X , Grizzle J W , Tabuada P . Control barrier function based quadratic programs for safety critical systems. IEEE Transactions on Automatic Control, 2017, 62(8): 3861–3876

[12] Bombile M , Billard A . Bimanual dynamic grabbing and tossing of objects onto a moving target. Robotics and Autonomous Systems, 2023, 167: 104481

[13] Gafur N, Yfantis V, Ruskowski M. Optimal scheduling and non-cooperative distributed model predictive control for multiple robotic manipulators. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague: IEEE, 2021, 390–397

[14] Beuke F, Alatartsev S, Jessen S, Verl A. Responsive and reactive dual-arm robot coordination. In: IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018, 316–322

[15] Wang Z , Gan Y , Dai X . Assembly-oriented task sequence planning for a dual-arm robot. IEEE Robotics and Automation Letters, 2022, 7(3): 8455–8462

[16] Yan L, Yang Y, Xu W, Vijayakumar S. Dual-arm coordinated motion planning and compliance control for capturing moving objects with large momentum. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid: IEEE, 2018, 7137–7144

[17] Yan L , Stouraitis T , Moura J , Xu W , Gienger M , Vijayakumar S . Impact-aware bimanual catching of large-momentum objects. IEEE Transactions on Robotics, 2024, 40: 2543–2563

[18] Zhao X , Zhang Y , Ding W , Tao B , Ding H . A dual-arm robot cooperation framework based on a nonlinear model predictive cooperative control. IEEE/ASME Transactions on Mechatronics, 2023, 29(5): 3993–4005

[19] Fan Y , Li X , Zhang K , Qian C , Zhou F , Li T , Huang Z . Learning robust skills for tightly coordinated arms in contact-rich tasks. IEEE Robotics and Automation Letters, 2024, 9(3): 2973–2980

[20] He Y , Wu M , Liu S . A distributed optimal control framework for multi-robot cooperative manipulation in dynamic environments. Journal of Intelligent & Robotic Systems, 2022, 105: 8

[21] Ren Y , Zhou Z , Xu Z , Yang Y , Zhai G , Leibold M , Ni F , Zhang Z , Buss M , Zheng Y . Enabling versatility and dexterity of the dual-arm manipulators: A general framework toward universal cooperative manipulation. IEEE Transactions on Robotics, 2024, 40: 2024–2045

[22] Khatib O . Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 1986, 5(1): 90–98

[23] Singh L, Stephanou H, Wen J. Real-time robot motion control with circulatory fields. In: IEEE International Conference on Robotics and Automation. Minneapolis: IEEE, 1996, 2737–2742

[24] Ataka A , Lam H K , Althoefer K . Reactive magnetic-field-inspired navigation method for robots in unknown convex 3-D environments. IEEE Robotics and Automation Letters, 2018, 3(4): 3583–3590

[25] Koptev M , Figueroa N , Billard A . Reactive collision-free motion generation in joint space via dynamical systems and sampling-based MPC. International Journal of Robotics Research, 2024, 43(13): 2049–2069

[26] Huber L , Billard A , Slotine J J . Avoidance of convex and concave obstacles with convergence ensured through contraction. IEEE Robotics and Automation Letters, 2019, 4(2): 1462–1469

[27] Jian Z, Yan Z, Lei X, Lu Z, Lan B, Wang X, Liang B. Dynamic control barrier function-based model predictive control to safety-critical obstacle-avoidance of mobile robot. In: IEEE International Conference on Robotics and Automation (ICRA). London: IEEE, 2023, 3679–3685

[28] Tang Y , Chu X , Huang J , Samuel Au K W . Learning-based MPC with safety filter for constrained deformable linear object manipulation. IEEE Robotics and Automation Letters, 2024, 9(3): 2877–2884

[29] Grandia R, Taylor A J, Ames A D, Hutter M. Multi-layered safety for legged robots via control barrier functions and model predictive control. In: IEEE International Conference on Robotics and Automation (ICRA). Xi’an: IEEE, 2021, 8352–8358

[30] Özgür E , Mezouar Y . Kinematic modeling and control of a robot arm using unit dual quaternions. Robotics and Autonomous Systems, 2016, 77: 66–73

[31] Funda J , Paul R P . A computational analysis of screw transformations in robotics. IEEE Transactions on Robotics and Automation, 1990, 6(3): 348–356

[32] Khatib O. Object manipulation in a multi-effector robot system. In: Proceedings of the 4th international symposium on Robotics Research. Santa Clara: MIT Press, 1988, 137–144

[33] Connolly T H, Pfeiffer F. Cooperating manipulator control using dual quaternion coordinates. In: the 33rd IEEE Conference on Decision and Control. Lake Buena Vista: IEEE, 1994, 2417–2418

[34] Caccavale F , Chiacchio P , Chiaverini S . Task-space regulation of cooperative manipulators. Automatica, 2000, 36(6): 879–887

[35] Quiroz-Omaña J J , Adorno B V . Whole-body control with (self) collision avoidance using vector field inequalities. IEEE Robotics and Automation Letters, 2019, 4(4): 4048–4053

[36] Adorno B V. Two-arm manipulation: From manipulators to enhanced human-robot collaboration. Dissertation for the Doctoral Degree. Montpellier: Université de Montpellier, 2011

[37] Liegeois A . Automatic supervisory control of the configuration and behavior of multibody mechanisms. IEEE Transactions on Systems, Man, and Cybernetics, 1977, 7(12): 868–871

[38] Chiaverini S . Singularity-robust task-priority redundancy resolution for real-time kinematic control of robot manipulators. IEEE Transactions on Robotics and Automation, 1997, 13(3): 398–410

RIGHTS & PERMISSIONS

Higher Education Press