Design of computer vision assisted machine learning based controller for the Stewart platform to track spatial objects

Abstract

The present work aims to develop an object tracking controller for the Stewart platform using a computer vision-assisted machine learning-based approach. This research is divided into two modules. The first module focuses on the design of a motion controller for the Physik Instrumente (PI)-based Stewart platform. In contrast, the second module deals with the development of a machine-learning-based spatial object tracking algorithm by collecting information from the Zed 2 stereo vision system. Presently, simple feed-forward neural networks (NN) are used to predict the orientation of the top table of the platform. While training, the x, y, and z coordinates of the three-dimensional (3D) object, extracted from images, are used as the input to the NN. In contrast, the orientation information of the platform (that is, rotation about the x, y, and z-axes) is considered as the output from the network. The orientation information obtained from the network is fed to the inverse kinematics-based motion controller (module 1) to move the platform while tracking the object. After training, the optimised NN is used to track the continuously moving 3D object. The experimental results show that the developed NN-based controller has successfully tracked the moving spatial object with reasonably good accuracy.

Graphical abstract

Keywords

Stewart platform / feed-forward neural networks / motion controller / inverse kinematics / stereo vision

References

[1] MittalR KNagrathI J. Robotics and Control. New York, NY: Tata McGraw-Hill, 2003

[2] Stewart D. A platform with six degrees of freedom. Proceedings of the Institution of Mechanical Engineers, 1965, 180(1): 371–386

[3] Liu K, Fitzgerald J M, Lewis F L, Member S. Kinematic analysis of a Stewart platform manipulator. IEEE Transactions on Industrial Electronics, 1993, 40(2): 282–293

[4] Zhou W, Chen W, Liu H. A new forward kinematic algorithm for a general Stewart platform. Mechanism and Machine Theory, 2015, 87: 177–190

[5] Chauhan D K S, Vundavilli P R. Forward kinematics of the Stewart parallel manipulator using machine learning. International Journal of Computational Methods, 2022, 2142009(8): 1–22

[6] Liu K, Lewis F, Lebret G, Taylor D. The singularities and dynamics of a Stewart platform manipulator. Journal of Intelligent & Robotic Systems, 1993, 8(3): 287–308

[7] Lebret G, Liu K, Lewis F L. Dynamic analysis and control of a Stewart platform manipulator. Journal of Robotic Systems, 1993, 10(5): 629–655

[8] Mostashiri N, Akbarzadeh A, Dhupia J. A comprehensive inverse dynamics problem of a Stewart platform by means of Lagrangian formulation. In: Proceedings of Dynamic Systems and Control Conference. New York, NY: American Society of Mechanical Engineers, 2017, 58271: V001T30A003

[9] Meng Q, Zhang T, He J F, Song J Y, Han J W. Dynamic modeling of a 6-degree-of-freedom Stewart platform driven by a permanent magnet synchronous motor. Frontiers of Information Technology & Electronic Engineering, 2010, 11: 751–761

[10] Tien T X, Cirkl D. Solution of inverse dynamics of Stewart−gough platform using substructure technique. Journal of Theoretical and Applied Mechanics, 2021, 59: 329–342

[11] Dasgupta B, Mruthyunjaya T S. A Newton−Euler formulation for the inverse dynamics of the Stewart platform manipulator. Mechanism and Machine Theory, 1998, 33(8): 1135–1152

[12] Liu M J, Li C X, Li C N. Dynamics analysis of the Gough−Stewart platform manipulator. IEEE Transactions on Robotics and Automation, 2000, 16(1): 94–98

[13] Jishnu A K, Chauhan D K S, Vundavilli P R. Design of neural network-based adaptive inverse dynamics controller for motion control of stewart platform. International Journal of Computational Methods, 2022, 19(8): 2142010

[14] Gosselin C M, Hadj-Messaoud A. Automatic planning of smooth trajectories for pick-and-place operations. In: Proceedings of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. New York, NY: American Society of Mechanical Engineers, 1991, 97768: 531–536

[15] Ömürlü V E, Yildiz I. A Stewart platform as a FBW flight control unit. Journal of Electrical Engineering, 2011, 62(4): 213–219

[16] Koszewnik A, Troc K, Słowik M. PID controllers design applied to positioning of ball on the Stewart platform. Acta Mechanicaet Automatica, 2014, 8(4): 214–218

[17] JosephI. Space debris: Understanding the risks to NASA spacecraft. 2023. Available at the website of NASA

[18] Wikipedia. Space debris. 2023. Available at the website of Wikipedia

[19] Ashith Shyam R B, Acharya M, Ghosal A. A heliostat based on a three degree-of-freedom parallel manipulator. Solar Energy, 2017, 157: 672–686

[20] Ashith Shyam R B, Ghosal A. Path planning of a 3-UPU wrist manipulator for sun tracking in central receiver tower systems. Mechanism and Machine Theory, 2018, 119: 130–141

[21] Tharamuttam J K, Ng A K. Design and development of an automatic solar tracker. Energy Procedia, 2017, 143: 629–634

[22] Baouche F Z, Abderezzak B, Ladmi A, Arbaoui K, Suciu G, Mihaltan T C, Raboaca M S, Hudișteanu S V, Țurcanu F E. Design and simulation of a solar tracking system for PV. Applied Sciences, 2022, 12(19): 9682

[23] Cammarata A. Optimized design of a large-workspace 2-DOF parallel robot for solar tracking systems. Mechanism and Machine Theory, 2015, 83: 175–186

[24] Liu X, Yang Y, Ma C, Li J, Zhang S. Real-time visual tracking of moving targets using a low-cost Unmanned Aerial Vehicle with a 3-axis stabilized gimbal system. Applied Sciences, 2020, 10(15): 5064

[25] Yilmaz A, Javed O, Shah M. Object tracking: A survey. ACM Computing Surveys, 2006, 38(4): 13

[26] Mandava R K, Vundavilli P R. Near optimal PID controllers for the biped robot while walking on uneven terrains. International Journal of Automation and Computing, 2018, 15(6): 689–706

[27] Khan M S, Mandava R K. A review on gait generation of the biped robot on various terrains. Robotica, 2023, 41(6): 1888–1930

[28] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[29] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345–359

[30] Regner D J, Salazar J D, Buschinelli P V, Machado M, Oliveira D, Santos J M, Marinho C A, Pinto T C. Object tracking control using a gimbal mechanism. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021, 43: 189–196

[31] Cong Danh N. The stability of a two-axis gimbal system for the camera. The Scientific World Journal, 2021, 2021: 2–9

Competing interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2024 Higher Education Press