Reinforcement learning-based attitude control for a quadrotor UAV system with performance constraints

Abstract

In this paper, a fuzzy logic-based fault-tolerant attitude control strategy is proposed for the attitude tracking of a quadrotor unmanned aerial vehicle (UAV) subject to actuator faults. The attitude dynamics of the quadrotor are represented using modified Rodrigues parameters. Inspired by the biological trial-and-error mechanism that reinforcement learning (RL) emulates, the proposed method is developed by integrating fuzzy logic systems (FLSs) with RL. To enhance the autonomous learning capability and tracking performance of the UAV system, actor–critic (AC) learning is introduced as an effective RL method. A cost function defined in terms of tracking errors is introduced, and an FLS is incorporated into the critic to approximate the cost function for performance evaluation. The actor is responsible for generating the control input based on the critic signals. Concurrently, another FLS is employed to approximate system uncertainties and actuator bias faults. Furthermore, to meet increasingly stringent control requirements, performance constraints are imposed to guarantee prescribed tracking performance. The system stability and convergence of tracking errors are analyzed using Lyapunov stability theory. Finally, simulations are conducted to verify the effectiveness of the proposed adaptive fault-tolerant attitude control scheme.

Keywords

Quadrotor UAV / reinforcement learning / actor-critic learning / fuzzy logic system / prescribed performance / actuator faults / adaptive fault-tolerant attitude control

References

[1] Huang H.,He W.,Chen Z.,Niu T.,Fu Q.. Development and experimental characterization of a robotic butterfly with a mass shifter mechanism Biomimetic Intell. Robot. 2022 2 100076

[2] He W.,Mu X.,Zhang L.,Zou Y.. Modeling and trajectory tracking control for flapping-wing micro aerial vehicles IEEE/CAA J. Autom. Sin. 2021 8 148 56

[3] Wu C.,Xiao Y.,Zhao J.,Cui F.,Wu X.,Liu W.. JingWei: a waterfowl-inspired flapping-wing robot with multimodal aerial-aquatic mobility IEEE Robot. Autom. Lett. 2025 10 11046 53

[4] Chen Y.,Pérez-Arancibia N. O.. Adaptive control of a VTOL uncrewed aerial vehicle for high-performance aerobatic flight Automatica 2024 159 109922

[5] Zheng Z.,Li J.,Guan Z.,Zuo Z.. Constrained moving path following control for UAV with robust control barrier function IEEE/CAA J. Autom. Sin. 2023 10 1557 70

[6] Xu B.,Suleman A.,Shi Y.. A multi-rate hierarchical fault-tolerant adaptive model predictive control framework: theory and design for quadrotors Automatica 2023 153 111015

[7] Dong F.,Yuan B.,Zhao X.,Ding Z.,Chen S.. Adaptive robust constraint-following control for morphing quadrotor UAV with uncertainty: a segmented modeling approach J. Franklin Inst. 2024 361 106678

[8] Zhao Z.,Zhang J.,Liu Z.,He W.,Hong K. S.. Adaptive quantized fault-tolerant control of a 2-DOF helicopter system with actuator fault and unknown dead zone Automatica 2023 148 110792

[9] Zhao W.,Liu H.,Lewis F. L.. Data-driven fault-tolerant control for attitude synchronization of nonlinear quadrotors IEEE Trans. Autom. Control 2021 66 5584 91

[10] Liu Y.,Dong X.,Shi P.,Ren Z.,Liu J.. Distributed fault-tolerant formation tracking control for multiagent systems with multiple leaders and constrained actuators IEEE Trans. Cybern. 2023 53 3738 47

[11] Ma Y.,Jiang B.,Wang J.,Gong J.. Adaptive fault-tolerant formation control for heterogeneous UAVs-UGVs systems with multiple actuator faults IEEE Trans. Aerosp. Electron. Syst. 2023 59 6705 16

[12] Hu Y.,Yan H.,Wang M.,Hu X.,Li Z.. Fuzzy observer-based input/output event-triggered control for Euler–lagrange systems with guaranteed performance and input saturation IEEE Trans. Fuzzy Syst. 2024 32 2077 88

[13] Ren Y.,Sun Y.,Liu Z.,Lam H. K.. Parameter-optimization-based adaptive fault-tolerant control for a quadrotor UAV using fuzzy disturbance observers IEEE Trans. Fuzzy Syst. 2025 33 593 605

[14] Kong L.,He W.,Yang C.,Li Z.,Sun C.. Adaptive fuzzy control for coordinated multiple robots with constraint using impedance learning IEEE Trans. Cybern. 2019 49 3052 63

[15] Zhang F.,Dai P.,Na J.,Gao G.,Shi Y.,Liu F.. Adaptive fuzzy tracking control for a class of uncertain nonlinear systems with improved prescribed performance IEEE Trans. Fuzzy Syst. 2025 33 1133 45

[16] Yu D.,Ma S.,Liu Y. J.,Wang Z.,Chen C. L. P.. Finite-time adaptive fuzzy backstepping control for quadrotor UAV with stochastic disturbance IEEE Trans. Autom. Sci. Eng. 2024 21 1335 45

[17] Su M.,Pu R.,Wang Y.,Yu M.. A collaborative siege method of multiple unmanned vehicles based on reinforcement learning Intell. Robot. 2024 4 39 60

[18] Dong L.,He Z.,Song C.,Sun C.. A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures J. Syst. Eng. Electron. 2023 34 439 59

[19] Zhang H.,He L.,Wang D.. Deep reinforcement learning for real-world quadrupedal locomotion: a comprehensive review Intell. Robot. 2022 2 275 97

[20] Wen G.,Yu D.,Zhao Y.. Optimized fuzzy attitude control of quadrotor unmanned aerial vehicle using adaptive reinforcement learning strategy IEEE Trans. Aerosp. Electron. Syst. 2024 60 6075 83

[21] Wen G.,Niu B.. Optimized distributed formation control using identifier–critic–actor reinforcement learning for a class of stochastic nonlinear multi-agent systems ISA Trans. 2024 155 1 10

[22] Han M.,Zhang L.,Wang J.,Pan W.. Actor-critic reinforcement learning for control with stability guarantee IEEE Robot. Autom. Lett. 2020 5 6217 24

[23] Ouyang Y.,Xue L.,Dong L.,Sun C.. Neural network-based finite-time distributed formation-containment control of two-Layer quadrotor UAVs IEEE Trans. Syst. Man Cybern. Syst. 2022 52 4836 48

[24] Ouyang Y.,Sun C.,Dong L.. Actor–critic learning based coordinated control for a dual-arm robot with prescribed performance and unknown backlash-like hysteresis ISA Trans. 2022 126 1 13

[25] Zhou Z. G.,Zhou D.,Chen X.,Shi X. N.. Adaptive actor-critic learning-based robust appointed-time attitude tracking control for uncertain rigid spacecrafts with performance and input constraints Adv. Space Res. 2023 71 3574 87

[26] Han H.,Cheng J.,Xi Z.,Lv M.. Symmetric actor–critic deep reinforcement learning for cascade quadrotor flight control Neurocomputing 2023 559 126789

[27] Yang S.,Pan Y.,Cao L.,Chen L.. Predefined-time fault-tolerant consensus tracking control for Multi-UAV systems with prescribed performance and attitude constraints IEEE Trans. Aerosp. Electron. Syst. 2024 60 4058 72

[28] Yu Z.,Li J.,Xu Y.,Zhang Y.,Jiang B.,Su C. Y.. Reinforcement learning-based fractional-order adaptive fault-tolerant formation control of networked fixed-wing UAVs with prescribed performance IEEE Trans. Neural Netw. Learn. Syst. 2024 35 3365 79

[29] Aforozi T. A.,Rovithakis G. A.. Prescribed performance tracking for uncertain MIMO pure-feedback systems with unknown and partially nonconstant control directions IEEE Trans. Autom. Control 2024 69 7285 92

[30] Wang X.,Kong L.,Meng T.,Xia J.,He W.. Event-triggered tracking control for a flapping-wing aerial vehicle with prescribed performance IEEE Trans. Aerosp. Electron. Syst. 2025 61 17476 87

[31] Li Z.,Wang X.,Guo H.,Xi L.,Liu G.,Li Y.. Distributed output feedback prescribed performance control for high-order nonlinear multi-agent systems IEEE Trans. Autom. Sci. Eng. 2025 22 12730 40

[32] Li D.,Ma G.,Li C.,He W.,Mei J.,Ge S. S.. Distributed attitude coordinated control of multiple spacecraft with attitude constraints IEEE Trans. Aerosp. Electron. Syst. 2018 54 2233 45

[33] Ouyang Y.,Dong L.,Wei Y.,Sun C.. Neural network based tracking control for an elastic joint robot with input constraint via actor-critic design Neurocomputing 2020 409 286 95

[34] Wang X.,Wang Q.,Sun C.. Prescribed performance fault-tolerant control for uncertain nonlinear MIMO system using actor–critic learning structure IEEE Trans. Neural Netw. Learn. Syst. 2022 33 4479 90

[35] Guo X.,Yan W.,Cui R.. Integral reinforcement learning-based adaptive NN control for continuous-time nonlinear MIMO systems with unknown control directions IEEE Trans. Syst. Man Cybern. Syst. 2020 50 4068 77