Deep Reinforcement Learning for automated scheduling of mining earthwork equipment with spatio-temporal safety constraints

Abstract

Large-scale machinery operated in a coordinated manner in earthworks for mining constitutes high safety risks. Efficient scheduling of such machinery, factoring in safety constraints, could save time and significantly improve the overall safety. This paper develops a model of automated equipment scheduling in mining earthworks and presents a scheduling algorithm based on deep reinforcement learning with spatio-temporal safety constraints. The algorithm not only performed well on safety parameters, but also outperformed randomized instances of various sizes set against real mining applications. Further, the study reveals that responsiveness to spatio-temporal safety constraints noticeably increases as the scheduling size increases. This method provides important noticeable improvements to safe automated scheduling in mining.

Graphical abstract

Keywords

deep reinforcement learning / mining earthwork / automated scheduling / spatio-temporal safety constraints

References

[1] AribiDDriss O BEl HaouziH B (2023). Multi-objective optimization of the dynamic and flexible job shop scheduling problem under workers fatigue constraints. Proceedings of ICAART (3), 301–308

[2] Ayhan B U, Tokdemir O B, (2020). Accident analysis for construction safety using latent class clustering and artificial neural networks. Journal of Construction Engineering and Management, 146( 3): 04019114–04019114

[3] Bo L, Li Z H, Liu Y, Yue Y G, Zhang Z H, Wang Y Y, (2022). Research on multi-level scheduling of mine water reuse based on improved whale optimization algorithm. Sensors, 22( 14): 5164–5164

[4] BothCDimitrakopoulos R J OEngineering (2020). Joint stochastic short-term production scheduling and fleet management optimization for mining complexes. 21(4): 1717–1743

[5] Choi G, Han S G, (2023). Reinforcement learning-based dynamic planning of cut and fill operations for earthwork optimization. Automation in Construction, 156: 105111

[6] Deepak M D, Mahesh G, (2023). A framework for enhancing construction safety through knowledge-based safety culture indicators. International Journal of Construction Management, 23( 12): 2039–2047

[7] DengNLi XSu Y J E S W oC (2020). Optimization of earthwork allocation path as vehicle route problem based on genetic algorithm. E3S Web of Conferences, 165(1): 04057

[8] Destouet C, Tlahig H, Bettayeb B, Mazari B J C, (2024). Multi-objective sustainable flexible job shop scheduling problem: Balancing economic, ecological, and social criteria. Computers & Industrial Engineering, 195: 110419

[9] FengZSha YLiuM (2023). Application of BIM in the design and construction of fabricated buildings. Computer-Aided Design and Applications

[10] Flores-Fonseca C, Linfati R, Escobar J W, (2022). Exact algorithms for production planning in mining considering the use of stockpiles and sequencing of power shovels in open-pit mines. Operations Research, 22( 3): 2529–2553

[11] Fu Y M, Li Z H, Chen N J, Qu C W, (2020). A discrete multi-objective rider optimization algorithm for hybrid flowshop scheduling problem considering makespan, Noise and Dust Pollution. IEEE Access: Practical Innovations, Open Solutions, 8: 88527–88546

[12] Gauthier S, Leduc M, Perfetto S J, Godwin A, (2022). Use of virtual reality to increase awareness of line-of-sight hazards around industrial equipment. Safety, 8( 3): 52–52

[13] Gondia A, Moussa A, Ezzeldin M, El-Dakhakhni W, (2023). Machine learning-based construction site dynamic risk models. Technological Forecasting and Social Change, 189: 122347

[14] Guo H Q, Nguyen H, Vu D A, Bui X N, (2021). Forecasting mining capital cost for open-pit mining projects based on artificial neural network approach. Resources Policy, 74: 101474

[15] Hallowell M R, Hinze J W, Baud K C, Wehle A, (2013). Proactive construction safety control: Measuring, monitoring, and responding to safety leading indicators. Journal of Construction Engineering and Management, 139( 10): 04013010–04013010

[16] Hong Y, Xie H Y, Agapaki E, Brilakis I, (2023). Graph-based automated construction scheduling without the use of BIM. Journal of Construction Engineering and Management, 149( 2): 05022020–05022020

[17] Huo D, Sari Y A, Kealey R, Zhang Q, (2023). Reinforcement learning-based fleet dispatching for greenhouse gas emission reduction in open-pit mining operations. Resources, Conservation and Recycling, 188: 106664

[18] Hwang S I, Son J H, Lee S H, (2014). development of scheduling model for earth work using genetic algorithm. KSCE Journal of Civil Engineering, 18( 6): 1618–1624

[19] Imam M, Baina K, Tabii Y, Ressami E, Adlaoui Y, Benzakour I, Abdelwahed E, (2023). The future of mine safety: A comprehensive review of anti-collision systems based on computer vision in underground mines. Sensors, 23( 9): 4294–4294

[20] InayahZCahyono Y HBudionoNDP(2023). Work shift system and heavy equipment operator fatigue in mine. 6(2): 223–229

[21] JoshiDPaithankar AChatterjeeSEqueenuddinS M J M (2022). Integrated parametric graph closure and branch-and-cut algorithm for open pit mine scheduling under uncertainty. 2(1): 32–51

[22] KulikovaE YBalovtsev S V (2020). Risk control system for the construction of urban underground structures. In: International Conference on Construction, Architecture and Technosphere Safety, Iccats 2020, 962

[23] LeeCChaM KimHChoH (2023). Detailed activity-based earthwork scheduling model to aid during the planning stage of road construction. Applied Sciences, 13(23)

[24] Lee D, Khan N, Park C, (2023). Rigorous analysis of safety rules for vision intelligence-based monitoring at construction jobsites. International Journal of Construction Management, 23( 10): 1768–1778

[25] Lee J H, Park S M, Kim S K, Kang L S, (2022). Linear 4D system using schedule-location charts for infrastructure projects. Automation in Construction, 141: 104413

[26] Lei K, Guo P, Zhao W C, Wang Y, Qian L M, Meng X Y, Tang L S, (2022). A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem. Expert Systems with Applications, 205: 117796

[27] Li D S, Lu M, (2019). Classical planning model-based approach to automating construction planning on earthwork projects. Computer-Aided Civil and Infrastructure Engineering, 34( 4): 299–315

[28] Liang Y, Liu Q X, (2022). Early warning and real-time control of construction safety risk of underground engineering based on building information modeling and internet of things. Neural Computing & Applications, 34( 5): 3433–3442

[29] Liu G W, Guo W Q, Fu E S, Yang C D, Li J M, (2023). Dynamic optimization of open-pit coal mine production scheduling based on ARIMA and fuzzy structured element. Frontiers of Earth Science, 10: 1040464

[30] LiuPXieM BianJLi HSongL (2020). A hybrid PSO–SVM model based on safety risk prediction for the design process in metro station construction. International Journal of Environmental Research and Public Health, 17(5): 1714

[31] Liu Y, You K, Jiang Y T, Wu Z G, Liu Z Y, Peng G, Zhou C, (2022). Multi-objective optimal scheduling of automated construction equipment using non-dominated sorting genetic algorithm (NSGA-III). Automation in Construction, 143: 104587

[32] Lu Y A, Qin W B, Zhou C, Liu Z H, (2023). Automated detection of dangerous work zone for crawler crane guided by UAV images via Swin Transformer. Automation in Construction, 147: 104744

[33] Lu Y A, You K, Zhou C, Chen J X, Wu Z A, Jiang Y T, Huang C, (2024). Video surveillance-based multi-task learning with swin transformer for earthwork activity classification. Engineering Applications of Artificial Intelligence, 131: 107814

[34] Manríquez F, González H, Morales N, (2023). Short-term open-pit production scheduling optimizing multiple objectives accounting for shovel allocation in stockpiles. Optimization and Engineering, 24( 1): 681–707

[35] Mirzaei-Nasirabad H, Mohtasham M, Askari-Nasab H, Alizadeh B, (2023). An optimization model for the real-time truck dispatching problem in open-pit mining operations. Optimization and Engineering, 24( 4): 2449–2473

[36] MohtashamMMirzaei-Nasirabad HAskari-NasabHAlizadehB J I J o MReclamation Environment (2022). Multi-stage optimization framework for the real-time truck decision problem in open-pit mines: A case study on Sungun copper mine. 36(7): 461–491

[37] Mu Y F, Yao T A, Jia H J, Yu X D, Zhao B, Zhang X S, Ni C W, Du L J, (2020). Optimal scheduling method for belt conveyor system in coal mine considering silo virtual energy storage. Applied Energy, 275: 115368

[38] Nikolaev A V, Vöth S, Kychkin A V, (2023). Application of the cybernetic approach to price-dependent demand response for underground mining enterprise electricity consumption. Journal of Mining Institute, 261: 403–414

[39] Ogunmodede O, Lamas P, Brickey A, Bogin G Jr, Newman A, (2022). Underground production scheduling with ventilation and refrigeration considerations. Optimization and Engineering, 23( 3): 1677–1705

[40] Ou G Z, Jiao Y Y, Zhang G H, Zou J P, Tan F, Zhang W S, (2021). Collapse risk assessment of deep-buried tunnel during construction and its application. Tunnelling and Underground Space Technology, 115: 104019

[41] Paithankar A, Chatterjee S, (2019). Open pit mine production schedule optimization using a hybrid of maximum-flow and genetic algorithms. Applied Soft Computing, 81: 105507

[42] Paithankar A, Chatterjee S, Goodfellow R, Asad M W A, (2020). Simultaneous stochastic optimization of production sequence and dynamic cut-off grades in an open pit mining operation. Resources Policy, 66: 101634

[43] Qin W B, Chen E J, Wang F, Liu W L, Zhou C, (2024). Data-driven models in reliability analysis for tunnel structure: A systematic review. Tunnelling and Underground Space Technology, 152: 105928

[44] QuinteroM F (2023). Analysis of the root causes of fatal injuries in the United States surface mines between 2008 and 2021

[45] Rachmawati T S N, Park H C, Kim S, (2022). A scenario-based simulation model for earthwork cost management using unmanned aerial vehicle technology. Sustainability, 15( 1): 503–503

[46] Sadeghi H, Mohandes S R, Hosseini M R, Banihashemi S, Mahdiyar A, Abdullah A, (2020). Developing an ensemble predictive safety risk assessment model: Case of malaysian construction projects. International Journal of Environmental Research and Public Health, 17( 22): 8395–8395

[47] Saurin T A, Formoso C T, Cambraia F B, (2008). An analysis of construction safety best practices from a cognitive systems engineering perspective. Safety Science, 46( 8): 1169–1183

[48] Shehadeh A, Alshboul O, Tatari O, Alzubaidi M A, Salama A H E S, (2022). Selection of heavy machinery for earthwork activities: A multi-objective optimization approach using a genetic algorithm. Alexandria Engineering Journal, 61( 10): 7555–7569

[49] StipanovicIBukhsh Z ARealeCGavinK (2021). A Multiobjective decision-making model for risk-based maintenance scheduling of railway earthworks. Applied Sciences, 11(3)

[50] WangG RYan X XKouZ MDengH SWangK S (2023). Research on operation conflict of auxiliary transport locomotive in complex mine based on extended petri net. Machines, 11(5)

[51] WangX YZhang M H (2022). How powerful are spectral graph neural networks. International Conference on Machine Learning, Vol 162

[52] WooSYeonJ JiMMoonI C ParkJ (2018). Deep reinforcement learning with fully convolutional neural network to solve an earthwork scheduling problem. In: 2018 Ieee International Conference on Systems, Man, and Cybernetics (Smc), 4230–4236

[53] Wu W J, Sun D H, Jin K Q, Sun Y, Si P B, (2022). Proximal policy optimization-based committee selection algorithm in blockchain-enabled mobile edge computing systems. China Communications, 19( 6): 50–65

[54] Xu X C, Gu X W, Wang Q, Zhao Y Q, Kong W Y, Zhu Z G, Wang F D, (2023). Enumeration optimization of open pit production scheduling based on mobile capacity search domain. Scientific Reports, 13( 1): 91–91

[55] Yan X X, Wang G R, Jiang K S, Kou Z M, Wang K S, Zhang L X, (2023). Multi-objective scheduling strategy of mine transportation robot based on three-dimensional loading constraint. Minerals, 13( 3): 431–431

[56] Yang S, Xu Z Intelligent scheduling for permutation flow shop with dynamic job arrival via deep reinforcement learning. Proceedings of 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), IEEE, 2672–2677

[57] You K, Zhou C, Ding L Y, (2023). Deep learning technology for construction machinery and robotics. Automation in Construction, 150: 104852

[58] ZamaniVTaghaddos HGholipour Y J A iC (2023). Sensor-fused simulation-based decision support system for earthmoving projects. Automation in Construction, 156, 105117

[59] Zhang X T, Guo A, Ai Y F, Tian B, Chen L, (2022). Real-time scheduling of autonomous mining trucks via flow allocation-accelerated tabu search. IEEE Transactions on Intelligent Vehicles, 7( 3): 466–479

[60] Zhou B H, Liao X M, (2020). Particle filter and Levy flight-based decomposed multi-objective evolution hybridized particle swarm for flexible job shop greening scheduling with crane transportation. Applied Soft Computing, 91: 106217

[61] Zhou C, Gao Y Y, Chen E J, Ding L Y, Qin W B, (2023). Deep learning technologies for shield tunneling: Challenges and opportunities. Automation in Construction, 154: 104982

[62] Zhou C, Gao Y Y, Zhou Y, She W, Shi Y S, Ding L Y, Miao C W, (2024). Properties and Characteristics of Regolith-Based Materials for Extraterrestrial Construction. Engineering, 37: 159–181

Competing Interests

The authors declare that they have no competing interests.

RIGHTS & PERMISSIONS

2025 Higher Education Press