Humanoid robot running through random stepping stones and jumping over obstacles: Step adaptation using spring-mass trajectories

Abstract

This study proposes a step adaptation framework for running through spring-mass trajectories and deadbeat control gain libraries. It includes four main parts: (1) Automatic spring-mass trajectory library generation; (2) Deadbeat control gain library generation through an actively controlled template model that resembles the whole-body dynamics well; (3) Trajectory selection policy development for step adaptation; (4) Mapping spring-mass trajectories to a humanoid model through a whole-body control (WBC) framework also accounting for closed-kinematic chain systems, self collisions, and reactive limb swinging. We show the inclusiveness and the robustness of the proposed framework through various challenging and agile behaviors such as running through randomly generated stepping stones, jumping over random obstacles, performing slalom motions, changing the running direction suddenly with a random leg, and rejecting significant disturbances and uncertainties through the MuJoCo physics simulator. We also perform additional simulations under a comprehensive set of uncertainties and noise to better justify the proposed method’s robustness against real-world challenges, such as signal noises, imprecision, modeling errors, and delays. All the aforementioned behaviors are performed with a single library and the same set of WBC control parameters without additional tuning. The spring-mass and the deadbeat control gain library are automatically computed in 4.5 s in total for 315 different trajectories.

Keywords

Humanoid and bipedal locomotion / Whole-body motion planning and control / Legged locomotion

CRediT authorship contribution statement

Sait Sovukluk: Writing – original draft, Methodology. Johannes Englsberger: Supervision. Christian Ott: Supervision.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 819358).

References

[1] R. Blickhan, The spring-mass model for running and hopping, J. Biomech. 22 (11–12) (1989) 1217-1227.

[2] D.E. Koditschek, R.J. Full, M. Buehler, Mechanical aspects of legged locomotion control, Arthropod Struct. Dev. 33 (3) (2004) 251-272.

[3] P. Holmes, R.J. Full, D. Koditschek, J. Guckenheimer, The dynamics of legged locomotion: Models, analyses, and challenges, SIAM Rev. 48 (2) (2006) 207-304.

[4] H. Geyer, A. Seyfarth, R. Blickhan, Compliant leg behaviour explains basic dynamics of walking and running, Proc. R. Soc. B: Biological Sci. 273 (1603) (2006) 2861-2867.

[5] J. Hurst, Walk this way: To be useful around people, robots need to learn how to move like we do, IEEE Spectr. 56 (3) (2019) 30-51.

[6] U. Saranli, W. Schwind, D. Koditschek, Toward the control of a multi-jointed, monoped runner, IEEE International Conference on Robotics and Automation, vol. 3, 1998, pp. 2676-2682.

[7] A. Seyfarth, H. Geyer, M. Günther, R. Blickhan, A movement criterion for running, J. Biomech. 35 (5) (2002) 649-655.

[8] M.M. Ankarali, U. Saranli, Stride-to-stride energy regulation for robust self-stability of a torque-actuated dissipative spring-mass hopper, Chaos: Interdiscip. J. Nonlinear Sci. 20 (3) (2010) 033121.

[9] G. Secer, U. Saranli, Control of planar spring–mass running through virtual tuning of radial leg damping, IEEE Trans. Robot. 34 (5) (2018) 1370-1383.

[10] M.H. Raibert, Legged Robots That Balance, MIT Press, (1986).

[11] C. Hubicki, A. Abate, P. Clary, S. Rezazadeh, M. Jones, A. Peekema, J. Van Why, R. Domres, A. Wu, W. Martin, H. Geyer, J. Hurst, Walking and running with passive compliance: Lessons from engineering: A live demonstration of the ATRIAS biped, IEEE Robot. Autom. Mag. 25 (3) (2018) 23-39.

[12] P.M. Wensing, D.E. Orin, High-speed humanoid running through control with a 3D-SLIP model, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 5134-5140.

[13] S. Sovukluk, J. Englsberger, C. Ott, Highly maneuverable humanoid running via 3D SLIP+foot dynamics, IEEE Robot. Autom. Lett. 9 (2) (2024) 1131-1138.

[14] F.M. Smaldone, N. Scianca, L. Lanari, G. Oriolo, From walking to running: 3D humanoid gait generation via MPC, Front. Robot. AI, 9 (2022).

[15] S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, R. Tedrake, Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot, Auton. Robots 40 (2016) 429-455.

[16] H. Dai, A. Valenzuela, R. Tedrake, Whole-body motion planning with centroidal dynamics and full kinematics, in: IEEE-RAS International Conference on Humanoid Robots, 2014, pp. 295-302.

[17] W.-L. Ma, A. Hereid, C.M. Hubicki, A.D. Ames, Efficient HZD gait generation for three-dimensional underactuated humanoid running, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, pp. 5819-5825.

[18] A. Tang, T. Hiraoka, N. Hiraoka, F. Shi, K. Kawaharazuka, K. Kojima, K. Okada, M. Inaba, HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation, in: IEEE International Conference on Robotics and Automation, 2024, pp. 13107-13114.

[19] S.H. Jeon, S. Heim, C. Khazoom, S. Kim, Benchmarking Potential Based Rewards for Learning Humanoid Locomotion, in: IEEE International Conference on Robotics and Automation, 2023, pp. 9204-9210.

[20] X. Meng, Z. Yu, X. Chen, Z. Huang, C. Dong, F. Meng, Online running-gait generation for bipedal robots with smooth state switching and accurate speed tracking, Biomimetics, 8 (1), (2023).

[21] R. Tajima, D. Honda, K. Suga, Fast running experiments involving a humanoid robot, in: 2009 IEEE International Conference on Robotics and Automation, 2009, pp. 1571-1576.

[22] T. Takenaka, T. Matsumoto, T. Yoshiike, S. Shirokura, Real time motion generation and control for biped robot -2nd report: Running gait pattern generation, -in: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009, pp. 1092-1099.

[23] Q. Zhang, P. Cui, D. Yan, J. Sun, Y. Duan, G. Han, W. Zhao, W. Zhang, Y. Guo, A. Zhang, R. Xu, Whole-body Humanoid Robot Locomotion with Human Reference, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, pp. 11225-11231.

[24] Y. Sakurai, T. Kamimura, Y. Sakamoto, S. Nishii, K. Sato, Y. Fujiwara, A. Sano, Bipedal robot running: human-like actuation timing using fast and slow adaptations, Adv. Robot. 38 (8) (2024) 577-588.

[25] F. Al-Hafez, G. Zhao, J. Peters, D. Tateo, LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion, in: 6th Robot Learning Workshop, NeurIPS, 2023.

[26] C. Tessler, Y. Guo, O. Nabati, G. Chechik, X.B. Peng, MaskedMimic: Unified physics-based character control through masked motion inpainting, ACM Trans. Graph., 43 (6), (2024).

[27] R.J. Griffin, G. Wiedebach, S. McCrory, S. Bertrand, I. Lee, J. Pratt, Footstep Planning for Autonomous Walking Over Rough Terrain, in: IEEE-RAS International Conference on Humanoid Robots, 2019, pp. 9-16.

[28] J. Englsberger, P. Kozłowski, C. Ott, A. Albu-Schäffer, Biologically inspired deadbeat control for running: From human analysis to humanoid control and back, IEEE Trans. Robot. 32 (4) (2016) 854-867.

[29] G. Mesesan, R. Schuller, J. Englsberger, C. Ott, A. Albu-Schäffer, Unified motion planner for walking, running, and jumping using the three-dimensional divergent component of motion, IEEE Trans. Robot. 2023, pp. 1-21.

[30] H. Geyer, Simple Models of Legged Locomotion Based on Compliant Limb Behavior, (Ph.D. thesis),, (2005).

[31] M. Ernst, H. Geyer, R. Blickhan, Spring-legged locomotion on uneven ground: a control approach to keep the running speed constant, Mobile Robotics: Solutions and Challenges, World Scientific, 2010, pp. 639-644.

[32] P.-B. Wieber, Holonomy and nonholonomy in the dynamics of articulated motion, Fast Motions in Biomechanics and Robotics: Optimization and Feedback Control, Springer, 2006, pp. 411-425.

[33] D.E. Orin, A. Goswami, S.-H. Lee, Centroidal dynamics of a humanoid robot, Auton. Robots 35 (2013) 161-176.

[34] P.M. Wensing, D.E. Orin, Improved computation of the humanoid centroidal dynamics and application for whole-body control, Int. J. Humanoid Robot. 13 (01) (2016) 1550039.

[35] P.M. Wensing, M. Posa, Y. Hu, A. Escande, N. Mansard, A. Del Prete, Optimization-based control for dynamic legged robots, IEEE Trans. Robot. (2023).

[36] B. Henze, M.A. Roa, C. Ott, Passivity-based whole-body balancing for torque-controlled humanoid robots in multi-contact scenarios, Int. J. Robot. Res. 35 (12) (2016) 1522-1543.

[37] B. Henze, A. Dietrich, C. Ott, An approach to combine balancing with hierarchical whole-body control for legged humanoid robots, IEEE Robot. Autom. Lett. 1 (2) (2016) 700-707.

[38] J. Englsberger, A. Dietrich, G.-A. Mesesan, G. Garofalo, C. Ott, A.O. Albu-Schäffer, MPTC-modular passive tracking controller for stack of tasks based control frameworks, Robot.: Sci. Syst. (RSS 2020), (2020).

[39] L. Sentis, O. Khatib, A whole-body control framework for humanoids operating in human environments, in: IEEE International Conference on Robotics and Automation, 2006, pp. 2641-2648.

[40] P.M. Wensing, D.E. Orin, Generation of dynamic humanoid behaviors through task-space control with conic optimization, in: 2013 IEEE International Conference on Robotics and Automation, 2013, pp. 3103-3109.

[41] O. Khatib, M. Jorda, J. Park, L. Sentis, S.-Y. Chung, Constraint-consistent task-oriented whole-body robot formulation: Task, posture, constraints, multiple contacts, and balance, Int. J. Robot. Res. 41 (13–14) (2022) 1079-1098.

[42] C. Ott, A. Dietrich, A. Albu-Schäffer, Prioritized multi-task compliance control of redundant manipulators, Automatica 53 (2015) 416-423.

[43] P.M. Wensing, Optimization and Control of Dynamic Humanoid Running and Jumping, (Ph.D. thesis),The Ohio State University, (2014).

[44] J. Englsberger, Combining Reduced Dynamics Models and Whole-Body Control for Agile Humanoid Locomotion, (Ph.D. thesis),Technische Universität München, (2016).

[45] S. Sovukluk, J. Englsberger, C. Ott, Whole Body Control Formulation for Humanoid Robots with Closed/Parallel Kinematic Chains: Kangaroo Case Study, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2023, pp. 10390-10396.

[46] S. Sovukluk, G. Zambella, T. Egle, C. Ott, Experimental Comparison of Whole-Body Control Formulations for Humanoid Robots in Task Acceleration and Task Force Spaces, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2025, pp. 20146-20153.

[47] S. Sovukluk, R. Schuller, J. Englsberger, C. Ott, Realtime Limb Trajectory Optimization for Humanoid Running Through Centroidal Angular Momentum Dynamics, in: IEEE International Conference on Robotics and Automation, 2025, pp. 404-410.

[48] E. Todorov, T. Erez, Y. Tassa, MuJoCo: A physics engine for model-based control, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 5026-5033.

[49] J. Carpentier, G. Saurel, G. Buondonno, J. Mirabel, F. Lamiraux, O. Stasse, N. Mansard, The Pinocchio C++ library – A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives, in: IEEE International Symposium on System Integrations, 2019.

[50] S. Agarwal, K. Mierle, T.C.S. Team, Ceres solver, (2023),URL https://github.com/ceres-solver/ceres-solver.