Dynamic Marshalling Resilient Control for Permanent Magnetic Maglev Trains Based on Optimized Twin Delayed Deterministic Policy Gradient

Abstract

This paper focuses on dynamic marshalling cooperative optimization control of permanent magnetic maglev trains. The traditional distributed controller is difficult to adjust the control parameters adaptively for complex and changeable tracking scenarios, resulting in an unsafe, inefficient and uncomfortable marshalling process. This paper proposes a distributed active disturbance rejection resilient controller based on optimized twin delayed deterministic policy gradient (TD3) algorithm. First, an improved distributed active disturbance rejection controller (DADRC) based on the bidirectional leader communication topology is designed to realize the multi-train dynamic marshalling cooperative control and its stability is proved theoretically. Second, different dynamic marshalling processes are adaptively optimized by using the TD3 algorithm to train the DADRC. Third, an adaptive Mayfly (AMA) algorithm with the Steffensen mutation mechanism is proposed to optimize some sensitive hyperparameters of the TD3 algorithm. The simulation results show that compared with the traditional distributed controller, the proposed resilient controller can adaptively adjust its own control parameters and flexibly optimize the dynamic marshalling process according to different tracking scenarios. Compared with the Mayfly-TD3 (MA-TD3) and AMA-deep deterministic policy gradient algorithms, the proposed AMA-TD3 algorithm shows more stable and fast convergence, and can achieve a successful control rate as high as 99.8% and a generalization rate of up to 93.2%.

Keywords

Permanent magnetic maglev train / Dynamic marshalling / Distributed active disturbance rejection resilient controller / Twin delayed deterministic policy gradient / Adaptive Mayfly algorithm

References

[1] QuagliettaE, WangM, GoverdeR. A multi-state train-following model for the analysis of virtual coupling railway operations. J Rail Transp Plan Manag, 2020, 15. 100195

[2] Flammini F, Marrone S, Nardone R et al (2018) Towards railway virtual coupling. In: Proceedings of the 2018 IEEE international conference on electrical systems for aircraft, railway, ship propulsion and road vehicles & international transportation electrification conference, Nottingham UK, Nov. 7–9. https://doi.org/10.1109/ESARS-ITEC.2018.8607523

[3] SchumannT. Increase of capacity on the shinkansen high-speed line using virtual coupling. Int J Transp Develop Integr, 2017, 1(4): 666-676.

[4] WilhelmiH, ThiemeT, HenningA, WagnerC. Aerodynamic investigations of the effects of virtual coupling on two next generation trains. New Res Numer Exp Fluid Mech XI, 2018, 136: 695-704.

[5] FadhelS, FromontS. Quantification of the aerodynamic gain on two virtually coupled modular trains. Int J Comput Method Exp Meas, 2022, 10(4): 372-384.

[6] Olfati-SaberR, FaxJ, MurrayR. Consensus and cooperation in networked multi-agent systems. Proc IEEE, 2007, 95(1): 215-233.

[7] MeoCD, VaioMD, FlamminiF, NardoneR, SantiniS, VittoriniV. ERTMS/ETCS virtual coupling: proof of concept and numerical analysis. IEEE Trans Intell Transp Syst, 2020, 21(6): 2545-2556.

[8] GuoZY, LiZQ. Virtual coupling of permanent magnetic maglev trains: An improved cooperative tracking and collision avoidance control protocol. IET Intel Transport Syst, 2023, 17(12): 2505-2518.

[9] GuoZY, LiZQ. Distributed auto disturbances rejection resilient control of permanent magnetic maglev trains based on the optimized deep deterministic policy gradient algorithm. IET Control Theory Appl, 2024, 18(11): 1383-1397.

[10] FelezJ, KimY, BorrelliF. A model predictive control approach for virtual coupling in railways. IEEE Trans Intell Transp Syst, 2019, 20(7): 2728-2739.

[11] FelezJ, Vaquero-SerranoM, de Dios SanzJ. A robust model predictive control for virtual coupling in train sets. Actuators, 2022, 1112372.

[12] WangD, CaoY. Adaptive cruise control of virtual coupled trains based on sliding mode. J Phys Conf Ser, 2022, 22241. 012109

[13] WangHW, ZhaoQQ, LinSY, et al.. A reinforcement learning empowered cooperative control approach for IIoT-based virtually coupled train sets. IEEE Trans Industr Inf, 2021, 17(7): 4935-4945.

[14] SuS, LiuWT, ZhuQY, et al.. A cooperative collision avoidance control methodology for virtual coupling trains. Accid Anal Prev, 2022, 173. 106703

[15] ZhangXR, YangDK, ZhangW, HuangJ. Optimal robust constraints-following control for rail vehicle virtual coupling. J Vib Control, 2023, 29(5): 1352-1365.

[16] WangB, YangDK, ZhangXR, et al.. Constraint-force driven control design for rail vehicle virtual coupling. J Vib Contro, 2022, 28(5): 551-563.

[17] ParkJ, LeeBH, EunY. Virtual coupling of railway vehicles: gap reference for merge and separation, robust control, and position measurement. IEEE Trans Intell Transp Syst, 2022, 23(2): 1085-1096.

[18] LiJX, WangHF, ZhangQH, ChaiM. Train dynamic coupling optimization control based on dynamic programming-sequential quadratic programming method. Railw Stand Des, 2022, 66(4): 168-174

[19] ZervoudakisK, TsafarakisS. A mayfly optimization algorithm. Comput Indus Eng, 2020, 145106559.

[20] LiuYFStable platoon control for virtually coupled train set under interference conditions, 2022, Beijing. Beijing Jiaotong University.

[21] LiZQ, YangH, LiuJM. Adaptive speed tracking control for high-speed electric multiple unit. J China Railw Soc, 2015, 4: 61-68

[22] WuQ, GeXH, HanQL, LiuYF. Railway virtual coupling: a survey of emerging control techniques. IEEE Trans Intell Vehic, 2023, 8(5): 1-17.

[23] LinY, WangCL. Computational model of safe braking curve of onboard ATP based on CBTC. J China Railw Soc, 2011, 8: 69-72

[24] DB23/T 3336-2022 (2022) Technical standard for suspended monorail transit. In: Local Standards of Heilongjiang Province.

[25] LinP, HuangY, ZhangQ, YuanZM. Distributed velocity and input constrained tracking control of high-speed train systems. IEEE Trans Sys Man Cybern Sys, 2021, 51(12): 7882-7888.

[26] GaoY, W, H, Wu, Z. J. Wang, . Cascaded linear active disturbance rejection control for uncertain systems with input constraints and output noise. Acta Automatica Sinica, 2022, 48(3): 843-852

[27] WangYY, ZhaoZL. Nonlinear active disturbance rejection attitude control for two-DOF unmanned helicopter. Acta Automatica Sinica, 2021, 47(8): 1951-1962

[28] HanY, LiSH. UAV path planning based on improved artificial potential field. Sys Eng Electron, 2007, 11(43): 3305-3311

[29] Sfei J, Saad M, Salish-Hassane H (2011) An improved artificial potential field approach to real-time mobile robot path planning in an unknown environment. In: 2011 IEEE inter symposium EOSE, Malaysia208-213

[30] MuDJ, HaoH, CaiH, et al.. Resarch on simulation of anti-disturbance characteristics of servo system based on tracking differentiator. Mech Electric Eng Technol, 2024, 53(5): 46-49

[31] ZhangWJ, LinZJ, ZhangRY. Design and applications of the maglev train suspension control algorithm via the improved tracking differentiator. J Railw Sc Eng, 2023, 20(10): 3954-3964

[32] ZhouY, WangM, AhnS. Distributed model predictive control approach for cooperative car-following with guaranteed local and string stability. Transp Res Part B, 2019, 128: 69-86.

[33] ZhouY, AhnS. Robust local and string stability for a decentralized car following control strategy for connected automated vehicles. Transp Res Part B, 2019, 125: 175-196.

[34] ZhangZY, HuangDY, HuangCX, et al.. TD3 algorithm improvement and autonomous vehicle parallel strategy learning. J Mech Eng, 2023, 59(8): 224-234

[35] PeiXF, MoSJ, ChenZF, et al.. Lane-changing research of autonomous vehicles based on TD3 algorithm in human-machine mixed driving traffic environment. China J Highway Transp, 2021, 34(11): 246-254

[36] YinHQ, WangC, YanC, XiangXJ, CaiBL, WeiCY. Deep reinforcement learning with multi-critic TD3 for decentralized multi-robot path planning. IEEE Trans Cognitive Develop Sys, 2024, 16(4): 1-15.

[37] CaoWW, ChenR, KangMX, LiYZ, ChaiTY. Signal compensation control with DDPG-Based tuning strategy for electronic throttle system. IEEE Trans Industr Electron, 2024, 71(3): 2924-2932.

[38] WangHB, FengLZ, ZhangYH, ZhouJT, DuHP. Human-machine authority allocation in indirect cooperative shared steering control with TD3 reinforcement learning. IEEE Trans Veh Technol, 2024, 73(6): 1-13.

[39] LiuTY, WangZ. Diversity controlled multiobjective particle swarm optimization. J Xidian Univers, 2021, 48(3): 106-114

[40] LiLF, LiuWD, LiL. Underwater magnetic field measurement error compensation based on improved mayfly algorithm. J Northwestern Poly-Tech Univers, 2022, 40(5): 1004-1011

[41] LiJY, MoYB, HongLL, GongR. Process control of chemical dynamic system based on multi-strategy mayfly optimization algorithm. J Intell Fuzzy Sys, 2024, 46(3): 7327-7352.

[42] AfzalinejadM. Rapidly convergent Steffensen based methods for unconstrained optimization. RAIRO Op Res, 2019, 53(2): 657-666.

Funding

National Key Research and Development Program of China(2023YFB4302100)

National Natural Science Foundation of China(52162048)

Jiangxi Provincial Program for Academic and Technical Leaders Training of Major Disciplines(20213BCJ22002)

Research Foundation of Education Bureau of Jiangxi Province, China(GJJ2200847)

RIGHTS & PERMISSIONS

The Author(s)