Temperature regulation of an optomechanical frame based on reinforcement learning active disturbance rejection control

Abstract

Spaceborne optomechanical systems face the dual challenges of extreme thermal disturbances and millikelvin-level temperature control precision during orbital operations, demanding robust control strategies. To address the performance limitations of conventional fixed-parameter active disturbance rejection control (ADRC) under complex operating conditions, this work proposes a Q-learning-enhanced adaptive ADRC framework. A thermal-transfer model incorporating multisource disturbances (solar radiation, structural conduction, and contact thermal resistance) is established, coupled with a reinforcement learning-driven parameter optimization mechanism. The ε-greedy policy dynamically adjusts observer bandwidth (ω_o ∈ [0.01, 0.2]) and controller bandwidth (ω_c ∈ [0.01, 0.1]) to enable real-time estimation and compensation of total disturbances. Simulation results demonstrate significant improvements over fixed-parameter ADRC and a self-tuning internal model control proportional-integral (SIMC-PI) controller: 31.3% and 15.4% reduction in settling time during setpoint responses, respectively; 21.8% lower integral absolute error (IAE) than the fixed-parameter ADRC during setpoint step responses; 12.7% and 52.5% enhancement in control precision over conventional fixed-parameter and SIMC-PI controllers, respectively, under ±10 K periodic and step thermal disturbances. Monte Carlo robustness tests reveal smaller fluctuation ranges of IAE, settling time, and overshoot under ±5% parameter perturbations. This methodology establishes a new paradigm for millikelvin-level thermal control in space optical payloads.

Keywords

optomechanical system / active disturbance rejection controller / Q-learning / high precision temperature control

References

[1] GAO J X, SONG Y S, LIU Y. Application of nonlinear PID self-immunity control in temperature control system of fast mirror[J]. Laser & Optoelectronics Progress, 2023, 60(5): 0523001. (in Chinese)

[2] WEN M X, LI J, WANG C, et al. Summary of high precision temperature sensing, measurement and control technology[J]. Acta Scientiarum Naturalium Universitatis Sunyatseni, 2021, 60(S1): 146-155. (in Chinese)

[3] YU F, XU N N, ZHAO Y, et al. Design and validation of thermal control system of Gaofen-4 satellite camera[J]. Space Return and Remote Sensing, 2016, 37(4): 72-79. (in Chinese)

[4] HIETA T, MERIMAA M. Spectroscopic measurement of air temperature[J]. International Journal of Thermophysics, 2010, 31(8): 1710-1718.

[5] AARON K M, HASHEMI A, MORRIS P A, et al. Space Interferometry Mission thermal design[C]// Astronomical Telescopes and Instrumentation. Waikoloa, HI, USA, 2003: 279.

[6] TONG Y L, LI G Q, GENG L Y. Current status of research on precision temperature control technology for spacecraft[J]. Space Return and Remote Sensing, 2016, 37(2): 1-8. (in Chinese)

[7] ZHAO Z M, LU P, SONG X Y. Design and validation of thermal control system for Gaofen-2 satellite camera[J]. Space Return and Remote Sensing, 2015, 36(4): 34-40. (in Chinese)

[8] GILMORE D. Spacecraft thermal control handbook, Volume Ⅰ: Fundamental technologies[M]. Washington, DC, USA: American Institute of Aeronautics and Astronautics, Inc., 2002.

[9] TONG Y L, LI G Q, YU L, et al. Application of PI control for precision temperature control of space camera[J]. Space Return and Remote Sensing, 2012, 33(4): 42-49. (in Chinese)

[10] DE PALO S, CAIROLA M, COMPASSI M, et al. Herschel heaters control modeling and correlation[J]. SAE International Journal of Aerospace, 2009, 4(1): 29-39.

[11] HAN J Q. From PID to active disturbance rejection control[J]. IEEE Transactions on Industrial Electronics, 2009, 56(3): 900-906.

[12] PAN C, YE Y, GU B Z, et al. Temperature control of the extinction cylinder of a 2.5 m large-field-of-view high-resolution telescope[J]. Infrared and Laser Engineering, 2023, 52(9): 20230024. (in Chinese)

[13] YUN Z R, WANG Z G, WANG J H. ADRC-based temperature control system for blackbody radiation sources[J]. Infrared Technology, 2019, 41(3): 232-238. (in Chinese)

[14] SIVAMAYIL K, RAJASEKAR E, ALJAFARI B, et al. A systematic study on reinforcement learning based applications[J]. Energies, 2023, 16(3): 1512.

[15] WILSON C, RICCARDI A. Improving the efficiency of reinforcement learning for a spacecraft powered descent with Q-learning[J]. Optimization and Engineering, 2023, 24(1): 223-255.

[16] YU B, LI C L, YANG T, et al. A high-precision temperature control method based on thermal characteristics of space camera[J]. Aerospace Return and Remote Sensing, 2014, 35(3): 84-89. (in Chinese)

[17] LI S. Research on high stability temperature control technology for optical machines[D]. Shanghai: University of Chinese Academy of Sciences, 2021. (in Chinese)

[18] ZHAO S, SHI H W, LIU X S, et al. Hydraulic servo flow control with third-order linear self-immunity controller[J]. Hydraulic and Pneumatic, 2021, 45(5): 149-156. (in Chinese)

[19] ZHAO X J, ZHU J, LUO X. Application of ADRC in lower limb rehabilitation training apparatus[J]. Journal of Southeast University (Natural Science Edition), 2019, 49(6): 1026-1032. (in Chinese)

[20] JIN H Y, SONG J C, LAN W Y, et al. On the characteristics of ADRC: A PID interpretation[J]. Science China Information Sciences, 2020, 63(10): 209201.

[21] WANG X P, ZHAO J, WANG B H, et al. Predictive current control system of PMSM based on LADRC[J]. Journal of Southeast University (English Edition), 2022, 38(3): 227-234.

[22] BAE Y, LEE S, YOON K J, et al. Three-dimensional dynamic modeling and transport analysis of solid oxide fuel cells under electrical load change[J]. Energy Conversion and Management, 2018, 165: 405-418.

[23] DAI W. Structural design and numerical simulation for high-precision sounding temperature sensor[J]. Transducer and Microsystem Technologies, 2022, 41(11): 5-8, 17. (in Chinese)

[24] CHENG D X, CHEN Z F, SU D W, et al. Stability analysis and robustness improvement of high-precision thermostat[J]. Journal of Hefei University of Technology (Natural Science Edition), 2022, 45(9): 1160-1164. (in Chinese)

[25] TARUN A K, CHUNDAWAT V S, MANDAL M, et al. Fast yet effective machine unlearning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 13046-13055.

[26] ZHANG J, JIANG X, SHI X Y, et al. Offline reinforcement learning for eco-driving control at signalized intersections[J]. Journal of Southeast University (Natural Science Edition), 2022, 52(4): 762-769. (in Chinese)

[27] ZHANG Y Q, LI D H. Active disturbance rejection control on a bubbling fluidized bed[J]. Journal of University of Science and Technology of China, 2012, 42(5): 391-397. (in Chinese)

Funding

National Key R&D Program of China(2022YFB3902902)

National Natural Science Foundation of China(52276003)