A physics-constrained autoencoder for full-waveform inversion using axial self-attention

Abstract

Full-waveform inversion (FWI) is highly sensitive to the initial model and low-frequency content, and it often suffers from cycle skipping and degraded resolution in complex media. We propose a physics-constrained autoencoder-based FWI with axial self-attention (AxPCAE-FWI). In a unified encoder-decoder architecture, a differentiable acoustic wave-equation solver is explicitly embedded, and the data-domain waveform misfit is used as the primary objective, so that training is consistently governed by wave physics and does not rely on paired seismic-velocity labels. The encoder extracts inversion-relevant, low-dimensional features, while the decoder reconstructs physically admissible velocity models. To capture long-range spatiotemporal dependencies in the time-offset plane, axial multi-head self-attention is introduced in the encoder, where global attention is computed separately along the time and receiver axes; two one-dimensional global attentions approximate a single two-dimensional (2D) global attention, reducing the computational complexity relative to full 2D attention while preserving global context. This design improves the representation of complex wavefield phenomena, including multiples, converted waves, and far-offset reflections, thereby alleviating cycle skipping when low-frequency information is limited. Numerical experiments on the Marmousi2 and Society of Exploration Geophysicists salt-dome models demonstrate stable convergence and high structural similarity with improved geological plausibility. Compared to conventional physics-informed adaptive extended FWI under the same iteration budget, AxPCAE-FWI yields clearer salt-body boundaries and better imaging of structurally complex regions, with improved robustness to noise.

Keywords

Full-waveform inversion / Deep learning / Self-attention / Physics-constrained autoencoder / Autoencoder

Acknowledgments

None.

Funding

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.

Author contributions

Conceptualization: Yunbo Niu, Yingming Qu, Zhenchun Li

Formal analysis: Yunbo Niu

Investigation: Yunbo Niu

Methodology: Yunbo Niu, Yingming Qu

Writing-original draft: Yunbo Niu

Writing-review & editing: Yunbo Niu, Yingming Qu, Zhenchun Li

Availability of data

All data are available upon reasonable request from the corresponding authors.

References

[1] Lailly P. The seismic inverse problem as a sequence of before stack migrations. In: Bednar JB, Redner R, Robinson E, Weglein A, editors. Conference on Inverse Scattering: Theory and Application. Society for Industrial and Applied Mathematics; 1983. p. 206-220.

[2] Tarantola A, Valette B. Generalized nonlinear inverse problems solved using the least squares criterion. Rev Geophys. 1982; 20(2):219-232. doi: 10.1029/RG020i002p00219

[3] Tarantola A. Inversion of seismic reflection data in the acoustic approximation. Geophysics. 1984; 49(8):1259-1266. doi: 10.1190/1.1441754

[4] Virieux J, Operto S. An overview of full-waveform inversion in exploration geophysics. Geophysics. 2009; 74(6):WCC1- WCC26. doi: 10.1190/1.3238367

[5] Mora P. Inversion=migration+tomography. Geophysics. 1989; 54(12):1575-1586. doi: 10.1190/1.1442625

[6] Xu S, Wang D, Chen F, Lambaré G, Zhang Y. Inversion of reflected seismic waves. In: SEG Technical Program Expanded Abstracts 2012. Las Vegas, NV. Society of Exploration Geophysicists; 2012. p. 1-7. doi: 10.1190/segam2012-1473.1

[7] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521(7553):436-444. doi: 10.1038/nature14539

[8] Kim Y, Nakata N. Geophysical inversion versus machine learning in inverse problems. Lead Edge. 2018; 37(12):894-901. doi: 10.1190/tle37120894.1

[9] Wu Y, McMechan GA, Wang Y. Adaptive feedback convolutional neural network-based high-resolution reflection-waveform inversion. J Geophys Res Solid Earth. 2022; 127(6):e2022JB024138. doi: 10.1029/2022JB024138

[10] Wang W, Yang F, Ma J. Velocity model building with a modified fully convolutional network. In: SEG Technical Program Expanded Abstracts 2018; SEG International Exposition and 88th Annual Meeting. Anaheim, CA: Society of Exploration Geophysicists; 2018. p. 2086-2090. doi: 10.1190/segam2018-2997566.1

[11] Wu Y, Lin Y, Zhou Z. InversionNet: Accurate and efficient seismic-waveform inversion with convolutional neural networks. In: SEG Technical Program Expanded Abstracts 2018; SEG International Exposition and 88th Annual Meeting. Anaheim, CA. Society of Exploration Geophysicists; 2018. p. 2096-2100. doi: 10.1190/segam2018-2998603.1

[12] Park MJ, Sacchi MD. Automatic velocity analysis using convolutional neural network and transfer learning. Geophysics. 2020; 85(1):V33-V43. doi: 10.1190/geo2018-0870.1

[13] Li S, Liu B, Ren Y, Chen Y. Deep-learning inversion of seismic data. IEEE Trans Geosci Remote Sens. 2020; 58(3):2135-2149. doi: 10.1109/TGRS.2019.2953473

[14] Wang W, Ma J. Velocity model building in a crosswell acquisition geometry with image-trained artificial neural networks. Geophysics. 2020; 85(2):U31-U46. doi: 10.1190/geo2018-0591.1

[15] Lu J, Wu C, Zhang H, Qu Y, Xu Q, Zhu M. Phi-Net: An Aggregation Network for Seismic Velocity Model Building. SSRN; 2024. Available from: https://ssrn.com/ abstract=5076729 Last accessed on 2025 Nov 29]. doi: 10.2139/ssrn.5076729

[16] Araya-Polo M, Jennings J, Adler A, Dahlke T. Deep-learning tomography. Lead Edge. 2018; 37(1):58-66. doi: 10.1190/tle37010058.1

[17] Song C, Geng H, Wang Y, et al. Simultaneous P- and S-wave seismic traveltime tomography using physics-informed neural networks. Geophys Prospect. 2025; 73(6):e70034. doi: 10.1111/1365-2478.70034

[18] Xue Z, Wang Y, Wu X, Ma J. Multi-geophysical information neural network for seismic tomography. Geophysics. 2025; 90(3): R89-R100. doi: 10.1190/geo2024-0039.1

[19] Zhang X, Wang Y, Zhang H. Rapid probabilistic seismic tomography using graph mixture density networks. J Geophys Res Solid Earth. 2025; 130(8):e2025JB031129. doi: 10.1029/2025JB031129

[20] Das V, Pollack A, Wollner U, Mukerji T. Convolutional neural network for seismic impedance inversion. Geophysics. 2019; 84(6):R869-R880. doi: 10.1190/geo2018-0838.1

[21] Wu B, Meng D, Wang L, Liu N, Wang Y. Seismic impedance inversion using fully convolutional residual network and transfer learning. IEEE Geosci Remote Sens Lett. 2020; 17(12):2140-2144. doi: 10.1109/LGRS.2019.2963106

[22] Taufik MH, Wang F, Alkhalifah T. Learned regularizations for multi-parameter elastic full waveform inversion using diffusion models. J Geophys Res Machine Learn Comput. 2024; 1(1):e2024JH000125. doi: 10.1029/2024JH000125

[23] Kazei V, Ovcharenko O, Plotnitskii P, Peter D, Zhang X, Alkhalifah T. Mapping full seismic waveforms to vertical velocity profiles by deep learning. Geophysics. 2021; 86(5):R711-R721. doi: 10.1190/geo2019-0473.1

[24] Yang F, Ma J. FWIGAN: Full-waveform inversion via a physics-informed generative adversarial network. J Geophys Res Solid Earth. 2023; 128(4):e2022JB025493. doi: 10.1029/2022JB025493

[25] Zhang Z, Lin Y. Data-driven seismic waveform inversion: A study on the robustness and generalization. IEEE Trans Geosci Remote Sens. 2020; 58(10):6900-6913. doi: 10.1109/TGRS.2020.2977635

[26] Yang Y, Engquist B, Sun J, et al. Application of optimal transport and the quadratic Wasserstein metric to full-waveform inversion. Geophysics. 2018; 83(1):R43-R62. doi: 10.1190/geo2016-0663.1

[27] Vantassel JP, Kumar K, Cox BR. Using convolutional neural networks to develop starting models for near-surface two-dimensional full waveform inversion. Geophys J Int. 2022; 231(1):72-90. doi: 10.1093/gji/ggac179

[28] Yang F, Ma J. Deep-learning inversion: A next-generation seismic velocity model building method. Geophysics. 2019; 84(4):R583-R599. doi: 10.1190/geo2018-0249.1

[29] Taufik MH, Huang X, Alkhalifah T. Latent representation learning in physics-informed neural networks for full waveform inversion. Earth Space Sci. 2025; 12(9):e2024EA004107. doi: 10.1029/2024EA004107

[30] Wu Y, McMechan GA. Parametric convolutional neural network-domain full-waveform inversion. Geophysics. 2019; 84(6):R881-R896. doi: 10.1190/geo2018-0224.1

[31] Zhu W, Xu K, Darve E, et al. Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification. Geophysics. 2022; 87(1):R93-R109. doi: 10.1190/geo2020-0933.1

[32] Wang Y, Jiang B, Wei Z, et al. Deep velocity generator: A plug-in network for FWI enhancement. IEEE Trans Geosci Remote Sens. 2023; 61:1-17. doi: 10.1109/TGRS.2023.3247880

[33] He Q, Wang Y. Reparameterized full-waveform inversion using deep neural networks. Geophysics. 2021; 86(1):V1-V13. doi: 10.1190/geo2019-0382.1

[34] Tromp J, Tape C, Liu Q. Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels. Geophys J Int. 2005; 160(1):195-216. doi: 10.1111/j.1365-246X.2004.02453.x