Adaptive compensation of nonlinear distortion in high-speed free-space optical systems

Weijie Wu; Yatao Yang

doi:10.1007/s11801-026-5027-y

Optoelectronics Letters ›› 2026, Vol. 22 ›› Issue (6) :360 -365. DOI: 10.1007/s11801-026-5027-y

Article

research-article

Adaptive compensation of nonlinear distortion in high-speed free-space optical systems

Weijie Wu ¹
, Yatao Yang ¹^,^a

Author information +

History +

PDF

Abstract

Nonlinear distortions from atmospheric turbulence and scattering critically degrade free-space optical (FSO) communication performance. We propose a hybrid Vol_LSTM framework combining the Volterra model’s nonlinear characterization with long short-term memory (LSTM) temporal dynamic modeling. This synergistic approach adaptively compensates atmospheric-induced distortions through parallel processing of instantaneous nonlinear effects and time-varying channel dynamics. Simulations and experiments demonstrate a 32 dB power budget with 31.53% performance improvement and 21% computational complexity reduction compared to conventional methods, alongside enhanced disturbance resilience. The framework’s dual mechanism of model-based Volterra filtering and data-driven LSTM adaptation provides a practical solution for atmospheric-challenged FSO systems, advancing robust optical communication design.

Keywords

Cite this article

Download citation ▾

Weijie Wu, Yatao Yang. Adaptive compensation of nonlinear distortion in high-speed free-space optical systems. Optoelectronics Letters, 2026, 22 (6) : 360-365 DOI:10.1007/s11801-026-5027-y

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Ray P P. A review on 6G for space-air-ground integrated network: key enablers, open challenges, and future direction. Journal of King Saud University - Computer and Information Sciences, 2022, 34(9): 6949-6976. J].

[2]	Alimi I A, Monteiro P P. Revolutionizing free-space optics: a survey of enabling technologies, challenges, trends, and prospects of beyond 5G free-space optical (FSO) communication systems. Sensors, 2024, 24(24): 8036. J].

[3]	Álvarez-Roa C, Álvarez-Roa M, Raddo T R, et al.. Cooperative terrestrial–underwater FSO system: design and performance analysis. Photonics, 2024, 11(1): 58. J].

[4]	Dhasarathan V, Singh M, Malhotra J. Development of high-speed FSO transmission link for the implementation of 5G and internet of things. Wireless networks, 2020, 26(4): 2403-2412. J].

[5]	Singh H, Mittal N, Miglani R, et al.. Design and analysis of high-speed free space optical (FSO) communication system for supporting fifth generation (5G) data services in diverse geographical locations of India. IEEE photonics journal, 2021, 13(5): 1-12. J].

[6]	Li L, Ji N, Wu Z, et al.. Design and experimental demonstration of an atmospheric turbulence simulation system for free-space optical communication. Photonics, 2024, 11(4): 334. J].

[7]	Liu X X, Li M, Liu Z. Research on FSO modulation classification algorithm based on deep learning. Optoelectronics letters, 2024, 20(12): 757-763. J].

[8]	Mallick K, Dutta B, Santra S, et al.. Nonlinearity optimization in FSO transport system and generation of 24-GHz MW signal based on OADM scheme and external modulation technique. Optical and quantum electronics, 2021, 53(9): 509. J].

[9]	Li L, Wang Z J. MZM nonlinear equalization by sinusoidal subcarrier modulation combined with LM-BP neural network. Optoelectronics letters, 2024, 20(10): 592-598. J].

[10]	Park Y J, Kim J Y, Jung J I. Predistortion approaches using coefficient approximation and bidirectional LSTM for nonlinearity compensation in visible light communication. Photonics, 2022, 9(3): 198. J].

[11]	Maghrabi M M T, Swaminathan H, Kumar S, et al.. Enhanced performance of artificial-neural-network-based equalization for short-haul fiber-optic communications. Sensors, 2023, 23(13): 5952. J].

[12]	Ali F, Afsar H, Alshamrani A, et al.. Machine learning-based mitigation of thermal and nonlinear impairments in optical communication grids. Optics & laser technology, 2025, 182: 112090. J].

[13]	Li L, Liu R, Sun Y. A nonlinear equalization for a PAM4 IM/DD system using MZM. Optoelectronics letters, 2022, 18(4): 238-242. J].

[14]	Elzanaty A, Alouini M S. Adaptive coded modulation for IM/DD free-space optical backhauling: a probabilistic shaping approach. IEEE transactions on communications, 2020, 68(10): 6388-6402. J].

[15]	Wettlin T, Calabro S, Rahman T, et al.. DSP for high-speed short-reach IM/DD systems using PAM. Journal of lightwave technology, 2020, 38(24): 6771-6778. J].

[16]	Tang X, Qiao Y, Chen Y W, et al.. Digital pre- and post-equalization for C-band 112-GB/s PAM4 short-reach transport systems. Journal of lightwave technology, 2020, 38(17): 4683-4690. J].

[17]	Xu Y, Huang L, Jiang W, et al.. Automatic optimization of Volterra equalizer with deep reinforcement learning for intensity-modulated direct-detection optical communications. Journal of lightwave technology, 2022, 40(16): 5395-5406. J].

[18]	Liu X, Li C, Jiang Z. Low-complexity pruned convolutional neural network based nonlinear equalizer in coherent optical communication systems. Electronics, 2023, 12(14): 3120. J].

[19]	Xie T, Sheng Q, Yu J. Deep learning equalizer connected with Viterbi-Viterbi algorithm for PAM D-band radio over fiber link. Sensors, 2023, 23(24): 9773. J].

[20]	Luo X, Bai C, Chi X, et al.. Nonlinear impairment compensation using transfer learning-assisted convolutional bidirectional long short-term memory neural network for coherent optical communication systems. Photonics, 2022, 9(12): 919. J].

[21]	Xie T, Yu J. 4Gbaud PS-16QAM D-band fiber-wireless transmission over 4.6 km by using balance complex-valued NN equalizer with random oversampling. Sensors, 2023, 23(7): 3655. J].

[22]	Zhou W, Shi J, Zhao L, et al.. Comparison of real- and complex-valued NN equalizers for photonics-aided 90-GBps D-band PAM-4 coherent detection. Journal of lightwave technology, 2021, 39(21): 6858-6868. J].

[23]	Lu X, Li Y, Chen J, et al.. Utilizing deep neural networks to extract non-linearity as entities in PAM visible light communication with noise. Optics express, 2022, 30(15): 26701-26715. J].

[24]	Liu X, Wang Y, Wang X, et al.. Bi-directional gated recurrent unit neural network based nonlinear equalizer for coherent optical communication system. Optics express, 2021, 29(4): 5923-5933. J].

[25]	Pan X, Wang X, Tian B, et al.. Machine-learning-aided optical fiber communication system. IEEE network, 2021, 35(4): 136-142. J].

[26]	Lu X, Lu C, Yu W, et al.. Memory-controlled deep LSTM neural network post-equalizer used in high-speed PAM VLC system. Optics express, 2019, 27(5): 7822-7833. J].

[27]	Lu X, Li Y, Chen X, et al.. Discrete wavelet transform assisted convolutional neural network equalizer for PAM VLC system. Optics express, 2024, 32(6): 10429-10443. J].