Enhancing Transient Performance of Propulsion Engines in Actual Sea Using Reinforcement Learning-Based Proportional-Integral-Derivative Gain Tuning

Abstract

The engine speed control system is critical to marine propulsion, particularly for ships navigating in wave conditions. Fluctuations in propeller torque, caused by wave-induced variations in the ship’s speed and direction, can lead to undesirable effects such as increased fuel consumption and greenhouse gas emissions, increased engine wear and tear, and degraded performance. In the field of marine diesel engines, a proportional—integral—derivative (PID) control algorithm is the most widely used method for speed regulation. Aiming to maintain a constant engine speed, the PID governor continuously adjusts the fuel injection into the engine cylinders in response to propeller torque fluctuations. The quality of control, and thus engine performance, depends on a set of PID control gains. However, these optimal PID gains are typically determined during engine testbed commissioning and remain fixed, which leads to suboptimal performance under varying operational conditions. While modelbased tuning techniques can achieve satisfactory results in relatively simple control tasks, particularly when the number of tunable parameters is small, adaptive tuning of controller parameters, while meeting multiple, often conflicting objectives, presents a considerable challenge. This paper explores the application of a reinforcement learning (RL) algorithm to adaptively tune PID gains, aiming to optimize engine performance under varying sea conditions and operational demands. A specific reward function is designed to effectively balance variations in engine shaft rotational speed and fuel pump index, ensuring optimal engine efficiency. The performance of the trust-region policy optimization and proximal policy optimization algorithms is evaluated for their suitability in continuous-action space control, demonstrating stable convergence and robustness. Simulation results indicate that the proposed RL algorithm outperforms the reference PID controller, offering promising results for adaptive gain scheduling in propulsion system control.

Keywords

Propulsion engine / Proportional—integral—derivative speed control / Adaptive tuning / Reinforcement learning

RIGHTS & PERMISSIONS

Harbin Engineering University and Springer-Verlag GmbH Germany, part of Springer Nature