With the further maturation of high speed digital-to-analog converters, super-channels using high spectrum efficiency technologies based on digital signal processing (DSP) at both transmitter and receiver sides has attracted a great deal of interest for the transmission of 100G beyond [
1–
3]. Technology options for transmission with higher bandwidth efficiency are being intensively studied, which comes in two major categories: reducing the spectrum bandwidth requirement [
1–
3] or increasing the modulation levels [
5–
7]. The former uses spectrum shaping technologies based on optical or electrical filtering, known as the Nyquist or super-Nyquist technology; the latter uses multi-level modulation formats, such as 32-QAM, 64-QAM and even higher order QAMs. However, both schemes relies on advanced DSP in transmitter or receiver due to the high sensitivity to the laser frequency offset, phase noise, intra-channel and inter-channel impairments, such as inter-symbol interference and inter-channel crosstalk.