Heuristic multi-agent reinforcement learning for imperfect maintenance policy optimization in multi-machine production systems with multi-state degradation
Yang WANG , Wei ZHANG , Junyong TAO , Guanghan BAI , Zhiqiang LIU , Li ZHANG
Eng. Manag ››
In multi-machine production systems with multi-state degradation, performance deterioration and unexpected failures can significantly reduce operational resilience and may even trigger cascading shutdowns. Therefore, developing an efficient maintenance policy has become essential for ensuring system reliability and economic performance. However, when the number of system components is large, traditional Markov decision process-based methods suffer from the exponential growth of the state and action spaces. As a result, the maintenance policy is difficult to obtain within an acceptable computational time. To address this challenge, this study develops a group-based multi-agent maintenance decision framework. The machines are divided into critical and non-critical groups, and machines within the same group share a group-specific state–action value representation. Building on the average-reward R-learning mechanism and the heuristic insight of HR-learning, this study further develops a Heuristic R-learning-based Multi-Agent Reinforcement Learning (HR-MARL) algorithm. The algorithm is designed to optimize long-run average performance while reducing unnecessary policy exploration. Finally, the effectiveness of the proposed method is validated through a serial-parallel system case study. The results show that the proposed method can effectively improve maintenance decision efficiency and policy quality for complex production systems, while maintaining strong robustness under different operating conditions.
multi-machine production systems with multi-state degradation / imperfect maintenance / heuristic multi-agent reinforcement learning
Higher Education Press 2026
/
| 〈 |
|
〉 |