Multi-Agent Reinforcement Learning Driven Dynamic Resource Optimisation in Healthcare Transportation Networks

Abstract

This paper presents HealthNet, a novel framework for the dynamic optimisation of healthcare transportation networks using multi-agent reinforcement learning. HealthNet leverages a spatiotemporal dependency module to capture complex spatiotemporal relationships in healthcare demand and resource allocation patterns, combined with centralised training and a decentralised execution approach. The system is modelled as a Markov game and solved using a deep reinforcement learning algorithm. Extensive simulations demonstrate that HealthNet outperforms eight state-of-the-art baseline methods across multiple network configurations and evaluation metrics. In a 4 × 4 grid network, HealthNet reduces average waiting times by 47.6% compared to model predictive control and 22.1% compared to the best-performing baseline. Traffic congestion rates are reduced to 16.7% compared to 42.3% for the worst baseline and 23.1% for the best baseline. Under irregular network topologies with stochastic disruptions, including demand surges and vehicle unavailability, HealthNet maintains superior performance with 42.1% lower average waiting time and 51.1% improvement in peak response times compared to competing approaches. These findings indicate that HealthNet can enhance both efficiency and resilience in healthcare transportation systems, potentially improving patient outcomes in complex urban environments.

Keywords

dynamic resource optimisation / healthcare transportation networks / reinforcement learning / spatiotemporal dependency / sustainable cities

Acknowledgements

This work is supported by the National Natural Science Foundation of China under No. 62202247.

Funding

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

Data available on request from the authors.

References

[1] J. Lv, A. Slowik, and K. Li, “Secure Output-Feedback Control of Transportation Cyber-Physical Systems for Emergency Medical Services Under Stealthy Attacks,” IEEE Transactions on Intelligent Transportation Systems 26, no. 9 (September 2025): 14179-14191, https://doi.org/10.1109/TITS.2024.3516937.

[2] O. Sabuncu and B. Bilgehan, “Revolutionizing Healthcare 5.0: Blockchain-Driven Optimization of Drone-to-Everything Communication Using 5G Network for Enhanced Medical Services,” Technology in Society 77 (June 2024): 102552, https://doi.org/10.1016/j.techsoc.2024.102552.

[3] J. Lv, A. Slowik, K. Li, and H. Jiang, “Distributed Edge Intelligence for Rapid In-Vehicle Medical Emergency Response in Internet-of-Vehicles,” IEEE Internet of Things Journal 12, no. 5 (March 2025): 4750-4760, https://doi.org/10.1109/JIOT.2024.3516947.

[4] J. Lv, S. Rani, and K. Li, “Intelligent Multi-level Network Optimization for Medical Logistics in Underground Transportation Systems: A Computational Intelligence Approach,” Computers & Industrial Engineering 209 (November 2025): 1-13, https://doi.org/10.1016/j.cie.2025.111451.

[5] Q. Wu, J. Fang, J. Zeng, J. Wen, and F. Luo, “Monte Carlo Simulation-Based Robust Workflow Scheduling for Spot Instances in Cloud Environments,” Tsinghua Science and Technology 29, no. 1 (February 2024): 112-126, https://doi.org/10.26599/tst.2022.9010065.

[6] J. H. Lv, B. G. Kim, B. D. Parameshachari, A. Slowik, and K. Q. Li, “Large Model-Driven Hyperscale Healthcare Data Fusion Analysis in Complex Multi-Sensors,” Information Fusion 115 (March 2025): 102780, https://doi.org/10.1016/j.inffus.2024.102780.

[7] M. Filipovska and H. S. Mahmassani, “Spatio-Temporal Characterization of Stochastic Dynamic Transportation Networks,” IEEE Transactions on Intelligent Transportation Systems 24, no. 9 (September 2023): 9929-9939, https://doi.org/10.1109/tits.2023.3276190.

[8] Y. Matsuo, Y. LeCun, M. Sahani, et al., Deep Learning, Reinforcement Learning, and World Models,” Neural Networks 152 (August 2022): 267-275, https://doi.org/10.1016/j.neunet.2022.03.037.

[9] Y. Liu, M. Y. Yang, and Z. G. Guo, “Reinforcement Learning Based Optimal Decision Making Towards Product Lifecycle Sustainability,” International Journal of Computer Integrated Manufacturing 35, no. 10-11 (November 2022): 1269-1296, https://doi.org/10.1080/0951192x.2022.2025623.

[10] H. C. Liu, Z. Y. Huang, X. Y. Mo, and C. Lv, “Augmenting Reinforcement Learning With Transformer-Based Scene Representation Learning for Decision-Making of Autonomous Driving,” IEEE Transactions on Intelligent Vehicles 9, no. 3 (March 2024): 4405-4421, https://doi.org/10.1109/tiv.2024.3372625.

[11] Y. Jiang, K. Di, R. Qian, et al., Optimizing Risk-Aware Task Migration Algorithm Among Multiplex UAV Groups Through Hybrid Attention Multi-Agent Reinforcement Learning,” Tsinghua Science and Technology 30, no. 1 (April 2024): 318-330, https://doi.org/10.26599/tst.2024.9010013.

[12] T. M. Hu, B. Luo, C. H. Yang, and T. W. Huang, “MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 10 (October 2023): 12098-12112, https://doi.org/10.1109/tpami.2023.3283537.

[13] C. Yu, J. M. Liu, S. M. Nemati, and G. S. Yin, “Reinforcement Learning in Healthcare: A Survey,” ACM Computing Surveys 55, no. 1 (January 2023): 1-36, https://doi.org/10.1145/3477600.

[14] D. J. Jagannath, R. J. Dolly, G. S. Let, and J. D. Peter, “An IoT Enabled Smart Healthcare System Using Deep Reinforcement Learning,” Concurrency and Computation: Practice and Experience 34, no. 28 (December 2022): e7403, https://doi.org/10.1002/cpe.7403.

[15] A. Z. Al-Marridi, A. Mohamed, and A. Erbad, “Optimized Blockchain-Based Healthcare Framework Empowered by Mixed Multi-Agent Reinforcement Learning,” Journal of Network and Computer Applications 224 (April 2024): 103834, https://doi.org/10.1016/j.jnca.2024.103834.

[16] S. Maheshwari, P. K. Jain, and K. Kotecha, “Route Optimization of Mobile Medical Unit With Reinforcement Learning,” Sustainability 15, no. 5 (March 2023): 3937, https://doi.org/10.3390/su15053937.

[17] A. A. Abdellatif, N. Mhaisen, A. Mohamed, A. Erbad, and M. Guizani, “Route Optimization of Mobile Medical Unit With Reinforcement Learning,” IEEE Internet of Things Journal 10, no. 24 (December 2023): 21982-22007, https://doi.org/10.1109/jiot.2023.3288050.

[18] E. Akin, “Deep Reinforcement Learning-Based Multirestricted Dynamic-Request Transportation Framework,” IEEE Transactions on Neural Networks and Learning Systems, Early Access (December 2023).

[19] H. R. Su, Y. D. Zhong, J. Y. J. Chow, B. Dey, and L. Jin, “EMVLight: A Multi-Agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System,” Transportation Research Part C: Emerging Technologies 146 (January 2023): 103955, https://doi.org/10.1016/j.trc.2022.103955.

[20] X. Fu, Q. F. Nie, X. B. Li, J. Liu, S. Nambisan, and S. Jones, “The Role of the Built Environment in Emergency Medical Services Delays in Responding to Traffic Crashes,” Journal of Transportation Engineering A 148, no. 10 (October 2022): 04022085, https://doi.org/10.1061/jtepbs.0000726.

[21] N. Bagheri, S. Yousefi, and G. Ferrari, “Software-Defined Traffic Light Preemption for Faster Emergency Medical Service Response in Smart Cities,” Accident Analysis & Prevention 196 (March 2024): 107425, https://doi.org/10.1016/j.aap.2023.107425.

[22] D. Noonan, M. Ryan, D. Whelan, and D. O'Neill, “Medical Fitness to Drive, Emergency Service Vehicles and Crash Risk,” Irish Journal of Medical Science 192, no. 5 (October 2023): 2487-2493, https://doi.org/10.1007/s11845-023-03301-0.

[23] T. Y. Zhao, Y. C. Tang, Q. M. Li, and J. Q. Wang, “Resilience-Oriented Network Reconfiguration Strategies for Community Emergency Medical Services,” Reliability Engineering & System Safety 231 (March 2023): 109029, https://doi.org/10.1016/j.ress.2022.109029.

[24] F. Abbracciavento, F. Zinnari, S. Formentin, A. G. Bianchessi, and S. M. Savaresi, “Multi-Intersection Traffic Signal Control: A Decentralized MPC-Based Approach,” IFAC Journal of Systems & Control 23 (March 2023): 100214, https://doi.org/10.1016/j.ifacsc.2022.100214.

[25] L. Z. Wang, Z. L. Ma, C. Y. Dong, and H. Wang, “Human-Centric Multimodal Deep (HMD) Traffic Signal Control,” IET Intelligent Transport Systems 17, no. 4 (April 2023): 744-753, https://doi.org/10.1049/itr2.12300.

[26] D. Yu, S. G. Wei, and L. G. Chai, “Traffic Signal Control in Mixed Traffic Environment Based on Advance Decision and Reinforcement Learning,” Transportation Safety and Environment 4, no. 4 (November 2022): tdac027, https://doi.org/10.1093/tse/tdac027.

[27] L. P. Yan, L. L. Zhu, K. Song, et al., Graph Cooperation Deep Reinforcement Learning for Ecological Urban Traffic Signal Control,” Applied Intelligence 53, no. 6 (March 2023): 6248-6265, https://doi.org/10.1007/s10489-022-03208-w.

[28] M. Y. Xu, T. Z. Qiu, J. Fang, H. Y. He, and H. T. Chen, “Signal-Control Refined Dynamic Traffic Graph Model for Movement-Based Arterial Network Traffic Volume Prediction,” Expert Systems with Applications 228 (October 2023): 120393, https://doi.org/10.1016/j.eswa.2023.120393.

[29] T. Xu, S. Barman, and M. W. Levin, “Smoothing-MP: A Novel Max-Pressure Signal Control Considering Signal Coordination to Smooth Traffic in Urban Networks,” Transportation Research Part C: Emerging 166 (September 2024): 104760, https://doi.org/10.1016/j.trc.2024.104760.

[30] S. J. Huang, C. N. Sun, R. Q. Wang, and D. Pompili, “Toward Adaptive and Coordinated Transportation Systems: A Multi-Personality Multi-Agent Meta-Reinforcement Learning Framework,” IEEE Transactions on Intelligent Transportation Systems 26, no. 8 (August 2025): 12148-12161, https://doi.org/10.1109/tits.2025.3560227.

[31] Y. H. Wang, Y. He, F. R. Yu, K. S. Wu, and S. Z. Chen, “Intelligence-Based Reinforcement Learning for Dynamic Resource Optimization in Edge Computing-Enabled Vehicular Networks,” IEEE Transactions on Mobile Computing 24, no. 3 (March 2025): 2394-2406, https://doi.org/10.1109/tmc.2024.3506161.

[32] W. X. Zhang, T. Zhao, Z. K. Zhao, Y. J. Wang, and F. R. Liu, “An Intelligent Strategy Decision Method for Collaborative Jamming Based on Hierarchical Multi-Agent Reinforcement Learning,” IEEE Transactions on Cognitive Communications and Networking 10, no. 4 (August 2024): 1467-1480, https://doi.org/10.1109/tccn.2024.3373640.

[33] Y. Ma, J. Q. Li, Y. F. Hu, and H. Chen, “A Battery Prognostics and Health Management Technique Based on Knee Critical Interval and Linear Complexity Self-Attention Transformer in Electric Vehicles,” IEEE Transactions on Intelligent Transportation Systems 25, no. 8 (August 2024): 10216-10230, https://doi.org/10.1109/tits.2024.3355436.

[34] A. Ala, V. Simic, D. Pamucar, and N. Bacanin, “Enhancing Patient Information Performance in Internet of Things-Based Smart Healthcare System: Hybrid Artificial Intelligence and Optimization Approaches,” Engineering Applications of Artificial Intelligence 131 (May 2024): 107889, https://doi.org/10.1016/j.engappai.2024.107889.

[35] X. Li, W. H. Xu, T. Q. Wang, and Y. Yuan, “Optimizing Integrated Eco-Driving Control and Holding Strategy for Real-Time Bus Bunching Mitigation,” IEEE Transactions on Intelligent Transportation Systems 25, no. 7 (July 2024): 7568-7582, https://doi.org/10.1109/tits.2024.3362345.