Toward autonomous mining: design and development of an unmanned electric shovel via point cloud-based optimal trajectory planning

Abstract

With the proposal of intelligent mines, unmanned mining has become a research hotspot in recent years. In the field of autonomous excavation, environmental perception and excavation trajectory planning are two key issues because they have considerable influences on operation performance. In this study, an unmanned electric shovel (UES) is developed, and key robotization processes consisting of environment modeling and optimal excavation trajectory planning are presented. Initially, the point cloud of the material surface is collected and reconstructed by polynomial response surface (PRS) method. Then, by establishing the dynamical model of the UES, a point to point (PTP) excavation trajectory planning method is developed to improve both the mining efficiency and fill factor and to reduce the energy consumption. Based on optimal trajectory command, the UES performs autonomous excavation. The experimental results show that the proposed surface reconstruction method can accurately represent the material surface. On the basis of reconstructed surface, the PTP trajectory planning method rapidly obtains a reasonable mining trajectory with high fill factor and mining efficiency. Compared with the common excavation trajectory planning approaches, the proposed method tends to be more capable in terms of mining time and energy consumption, ensuring high-performance excavation of the UES in practical mining environment.

Graphical abstract

Keywords

autonomous excavation / unmanned electric shovel / point cloud / excavation trajectory planning

References

[1] Wei B C , Gao F . A method to calculate working capacity space of multi-DOF manipulator and the application in excavating mechanism. Frontiers of Mechanical Engineering, 2012, 7( 2): 109– 119

[2] Wei B , Gao F , Chen J , He J , Zhao X . A method for selecting driving system parameters of a new electric shovel’s excavating mechanism with three-DOF. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2011, 225( 11): 2661– 2672

[3] Wu J Q , Wang G Q , Bi Q S , Hall R . Digging force and power consumption during robotic excavation of cable shovel: experimental study and DEM simulation. International Journal of Mining, Reclamation and Environment, 2021, 35( 1): 12– 33

[4] Awuah-Offei K , Frimpong S . Numerical simulation of cable shovel resistive forces in oil sands excavation. International Journal of Mining, Reclamation and Environment, 2006, 20( 3): 223– 238

[5] Stavropoulou M , Xiroudakis G , Exadaktylos G . Analytical model for estimation of digging forces and specific energy of cable shovel. Coupled Systems Mechanics, 2013, 2( 1): 23– 51

[6] Rasuli A , Tafazoli S , Dunford W G . Dynamic modeling, parameter identification, and payload estimation of mining cable shovels. In: Proceedings of 2014 IEEE Industry Application Society Annual Meeting. Vancouver: IEEE, 2014, 1– 9

[7] Shekhar R C , Maciejowski J M . Surface excavation with model predictive control. In: Proceedings of the 49th IEEE Conference on Decision and Control (CDC). Atlanta: IEEE, 2010, 5239– 5244

[8] Awuah-Offei K , Frimpong S . Efficient cable shovel excavation in surface mines. Geotechnical and Geological Engineering, 2011, 29( 1): 19– 26

[9] Patnayak S , Tannant D D . Performance monitoring of electric cable shovels. International Journal of Surface Mining, Reclamation and Environment, 2005, 19( 4): 276– 294

[10] Frimpong S , Li Y . Stress loading of the cable shovel boom under in-situ digging conditions. Engineering Failure Analysis, 2007, 14( 4): 702– 715

[11] Li Y , Frimpong S . Hybrid virtual prototype for analyzing cable shovel component stress. The International Journal of Advanced Manufacturing Technology, 2008, 37( 5–6): 423– 430

[12] Frimpong S , Hu Y F . Intelligent cable shovel excavation modeling and simulation. International Journal of Geomechanics, 2008, 8( 1): 2– 10

[13] Song X G , Zhang T C , Yuan Y L , Wang X B , Sun W . Multidisciplinary co-design optimization of the structure and control systems for large cable shovel considering cross-disciplinary interaction. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2020, 234( 22): 4353– 4365

[14] Osa T , Aizawa M . Deep reinforcement learning with adversarial training for automated excavation using depth images. IEEE Access: Practical Innovations, Open Solutions, 2022, 10 : 4523– 4535

[15] Yoshida H , Yoshimoto T , Umino D , Mori N . Practical full automation of excavation and loading for hydraulic excavators in indoor environments. In: Proceedings of 2021 IEEE the 17th International Conference on Automation Science and Engineering (CASE). Lyon: IEEE, 2021, 2153– 2160

[16] Zhang L J , Zhao J X , Long P X , Wang L Y , Qian L F , Lu F X , Song X B , Manocha D . An autonomous excavator system for material loading tasks. Science Robotics, 2021, 6( 55): eabc3164

[17] Phillips T G Green M E McAree P R. Is it what I think it is? Is it where I think it is? Using point-clouds for diagnostic testing of a digging assembly’s form and pose for an autonomous mining shovel. Journal of Field Robotics, 2016, 33(7): 1013– 1033

[18] Zhang T C , Fu T , Song X G , Qu F Z . Multi-objective excavation trajectory optimization for unmanned electric shovels based on pseudospectral method. Automation in Construction, 2022, 136 : 104176

[19] Phillips T , Hahn M , McAree R . An evaluation of ranging sensor performance for mining automation applications. In: Proceedings of 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Wollongong: IEEE, 2013, 1284– 1289

[20] Phillips T G , Guenther N , McAree P R . When the dust settles: the four behaviors of LiDAR in the presence of fine airborne particulates. Journal of Field Robotics, 2017, 34( 5): 985– 1009

[21] Green M E , Ridley A N , McAree P R . Pose verification for autonomous equipment interaction in surface mining. In: Proceedings of 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Wollongong: IEEE, 2013, 1199– 1204

[22] D’Adamo T A , Phillips T G , McAree P R . Registration of three-dimensional scanning LiDAR sensors: an evaluation of model-based and model-free methods. Journal of Field Robotics, 2018, 35( 7): 1182– 1200

[23] Dunbabin M , Corke P . Autonomous excavation using a rope shovel. Journal of Field Robotics, 2006, 23( 6–7): 379– 394

[24] Awuah-Offei K , Frimpong S . Cable shovel digging optimization for energy efficiency. Mechanism and Machine Theory, 2007, 42( 8): 995– 1006

[25] Bi Q S , Wang G Q , Wang Y P , Yao Z W , Hall R . Digging trajectory optimization for cable shovel robotic excavation based on a multi-objective genetic algorithm. Energies, 2020, 13( 12): 3118

[26] Jud D , Leemann P , Kerscher S , Hutter M . Autonomous free-form trenching using a walking excavator. IEEE Robotics and Automation Letters, 2019, 4( 4): 3208– 3215

[27] Son B , Kim C U , Kim C , Lee D . Expert-emulating excavation trajectory planning for autonomous robotic industrial excavator. In: Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas: IEEE, 2020, 2656– 2662

[28] Lee D , Jang I , Byun J , Seo H , Kim H J . Real-time motion planning of a hydraulic excavator using trajectory optimization and model predictive control. In: Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague: IEEE, 2021, 2135– 2142

[29] Wang X B , Sun W , Li E , Song X G . Energy-minimum optimization of the intelligent excavating process for large cable shovel through trajectory planning. Structural and Multidisciplinary Optimization, 2018, 58( 5): 2219– 2237

[30] Wang X B , Song X G , Sun W . Surrogate based trajectory planning method for an unmanned electric shovel. Mechanism and Machine Theory, 2021, 158 : 104230

[31] Nurunnabi A , West G , Belton D . Robust locally weighted regression techniques for ground surface points filtering in mobile laser scanning three dimensional point cloud data. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54( 4): 2181– 2193

[32] Holz D , Ichim A E , Tombari F , Rusu R B , Behnke S . Registration with the point cloud library: a modular framework for aligning in 3-D. IEEE Robotics & Automation Magazine, 2015, 22( 4): 110– 124

[33] Durovsky F . Point cloud based bin picking: object recognition and pose estimation using region growing segmentation algorithm. Applied Mechanics and Materials, 2015, 791 : 189– 194

[34] Jain K , Pannu H S . Autonomic point cloud-based surface reconstruction using SVR. The Imaging Science Journal, 2018, 66( 1): 59– 67

[35] Slabanja J , Meden B , Peer P , Jaklič A , Solina F . Segmentation and reconstruction of 3D models from a point cloud with deep neural networks. In: Proceedings of 2018 International Conference on Information and Communication Technology Convergence (ICTC). Jeju: IEEE, 2018, 118– 123

[36] Pan R J , Skala V . A two-level approach to implicit surface modeling with compactly supported radial basis functions. Engineering with Computers, 2011, 27( 3): 299– 307

[37] Zhao S , Lu T F , Koch B , Hurdsman A . Dynamic modelling of 3D stockpile for life-cycle management through sparse range point clouds. International Journal of Mineral Processing, 2013, 125 : 61– 77

[38] Nazemizadeh M , Rahimi H N , Amini Khoiy K . Trajectory planning of mobile robots using indirect solution of optimal control method in generalized point-to-point task. Frontiers of Mechanical Engineering, 2012, 7( 1): 23– 28

[39] Powell M J D. A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez S, Hennart J P, eds. Advances in Optimization and Numerical Analysis. Mathematics and Its Applications, vol 275. Dordrecht: Springer, 1994

[40] Panda B , Garg A , Jian Z , Heidarzadeh A , Gao L . Characterization of the tensile properties of friction stir welded aluminum alloy joints based on axial force, traverse speed, and rotational speed. Frontiers of Mechanical Engineering, 2016, 11( 3): 289– 298

[41] Rossi C , Savino S . Robot trajectory planning by assigning positions and tangential velocities. Robotics and Computer-Integrated Manufacturing, 2013, 29( 1): 139– 156

Nomenclature

Abbreviations
GNSS	Global navigation satellite system
IMU	Inertial measurement unit
LS	Logarithmic spiral
MSoE	Maximum sum of evidence
PID	Proportional integral derivative
PLC	Programmable Logic Controller
PRS	Polynomial response surface
PTP	Point to point
RBF	Radial basis functions
TVP	Trapezoidal velocity profile
UES	Unmanned electric shovel
Variables
a1, a2	Acceleration in uniform acceleration and deceleration stages, respectively
ay	Excavation acceleration in the y direction
ay1	Acceleration in uniform acceleration stage and uniform deceleration stage in the y direction
az	Excavation acceleration in the z direction
az1	Acceleration in uniform acceleration stage and uniform deceleration stage in the z direction
c	Polynomial trajectory coefficient
cy6, cz6	Six-degree polynomial coefficients in the y andz directions, respectively
Ci	Constraint in trajectory planning
D	Vandermond matrix
Dxy	Projection area in the horizontal direction when the dipper teeth cut the material surface
Ec	Energy consumption of the crowd machinery
Eh	Energy consumption of the hoist machinery
Eper	Energy consumption per volume
fs	Polynomial function
Fc	Crowd force
Fca	Maximum allowable value of the crowd force
ftr(x, y)	Excavation trajectory
F_{\text{c}max}	Maximum crowd force
F_{\text{h}}	Hoist force
F_{\text{ha}}	Maximum allowable value of the hoist force
F_{\text{h}max}	Maximum hoist force
F_i	Generalized force
F_{\text{n}}	Normal excavation resistance
F_{\text{t}}	Tangential excavation resistance
g	Acceleration of gravity
h_{\text{b}min}	Minimum vertical height of the dipper bottom
h_{\text{mf}}	Material height corresponding to the final position of the excavation trajectory
h_{\epsilon }	Margin height
J	Objective function
J_1	Mining efficiency
J_2	Mining production
J_3	Energy consumption
J_{max}	Upper bound
J_{min}	Lower bound
k	Polynomial order
L	Lagrange function
L_{\text{d}}	Length of the dipper
L_{\text{h}}	Length of the dipper handle
L_{\text{s}}	Loss function
m_0	Mass of the empty dipper
m_{\text{d}}	Total mass of the dipper
m_{\text{h}}	Mass of the dipper handle
m_{\text{m}}	Mass of the loaded material
n	Degree of the polynomial
N	Number of points
p_y	Position of the excavation trajectory in the y direction
p_y(t_{\text{f}})	Final position of the excavation trajectory in the y direction
p_{\mathit{\text{z}}}	Position of the excavation trajectory in the z direction
p_{\mathit{\text{z}}}(t_{\text{f}})	Final position of the excavation trajectory in the z direction
\mathit{P}	Point cloud
P_{\text{ca}}	Maximum allowable value of the crowd power
P_{\text{c}max}	Maximum crowd power
P_{\text{ha}}	Maximum allowable value of the hoist power
P_{\text{h}max}	Maximum hoist power
q_i	Generalized coordinate
r	Stretching length of the dipper handle
r_{\text{a}}	Maximum allowable value of the stretching length of the dipper handle
r_{max}	Maximum stretching length of the dipper handle
\dot{r}(t)	Velocity of the dipper handle
{\dot{r}}_{max}	Maximum velocity of the dipper
{\dot{r}}_{min}	Minimum dipper handle velocity
s	Degree of freedom
t	Time
t_0	Initial time in excavation
t_1	Switching time between uniform acceleration and uniform stage
t_2	Switching time between uniform stage and uniform deceleration stage
t_{\text{f}}	Final time in excavation
t_{y1}	Switching time between uniform acceleration and uniform stage in the y direction
t_{y2}	Switching time between uniform stage and uniform deceleration stage in the y direction
t_{\mathit{\text{z}}1}	Switching time between uniform acceleration and uniform stage in the z direction
t_{\mathit{\text{z}}2}	Switching time between uniform stage and uniform deceleration stage in the z direction
v_0	Initial velocity
v_{\text{ha}}	Maximum allowable velocity of the dipper handle
v_{\text{ra}}	Maximum allowable value of the rope velocity
v_{\text{r}max}	Maximum velocity of the hoist rope
v_{\text{r}min}	Minimum rope velocity
v_{\text{u}}	Velocity in uniform stage
v_y	Excavation velocity in the y direction
v_{\mathit{\text{z}}}	Excavation velocity in the z direction
v_{\text{r}}(t)	Rope velocity
V	Loaded volume
V_{\text{n}}	Nominal load capacity
x	Coordinate of the point in the x direction
x	Coordinate vector of the point cloud in the x direction
y	Coordinate of the point in the y direction
y	Coordinate vector of the point cloud in the y direction
{\mathit{\text{z}}}_i	True value of the sample i
\overline{\mathit{\text{z}}}	Mean value for all samples
\widehat{\mathit{\text{z}}}	Z-direction response variable
{\widehat{\mathit{\text{z}}}}_i	Prediction value of the sample i
\mathit{z}	Coordinate vector of the point cloud in the z direction
\hat{\mathit{z}}	Prediction value of z
\mathit{\beta }	Coefficient column vector
{\beta }_{ij}	Coefficient of the polynomial function
\delta	Cutting angle
\theta	Angle between the vertical direction and the axis of the dipper handle
{\theta }_{\text{p}}	Polar angle
\rho	Material density
{\rho }_0	Initial polar diameter
{\omega }_1, {\omega }_2, {\omega }_3	Weight coefficients
\varpi	Angle between the hoist rope and the dipper handle

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 52075068) and the Science and Technology Major Project of Shanxi Province, China (Grant No. 20191101014).

RIGHTS & PERMISSIONS

2022 Higher Education Press