Fully automatic differentiation with coupling deep neural networks for full-waveform inversion

Abstract

Seismic full-waveform inversion (FWI) is a powerful technique used in geophysical exploration to infer subsurface properties. However, FWI often suffers from challenges such as cycle skipping and sensitivity to uncertainties in seismic observations. This study aims to tackle these challenges by developing a novel fully automatic differentiation (AD) strategy for seismic FWI, coupling U-Net-based reparameterization inspired by the deep image prior concept into a reformulated wave equation simulation framework utilizing recurrent neural networks (RNNs). We demonstrate that the U-Net reparameterization serves as a form of implicit regularization for FWI, mitigating the ill-posed nature of the inversion problem and enhancing the stability of the optimization process. In addition, the RNN reformulation offers a flexible approach for backpropagating the FWI misfit, allowing the gradient with respect to the velocity parameters to be computed using the AD capabilities inherent in deep learning frameworks. Through extensive experiments on synthetic datasets, we showcase the regularization effect of our proposed method, leading to improved inversion results in terms of accuracy and robustness. This study offers a promising avenue for enhancing the reliability and accuracy of FWI through the lens of deep learning methodologies.

Keywords

Full-waveform inversion / U-Net / Deep image prior / RNN-based FWI

Funding

This work was financially supported by the National Natural Science Foundation of China Program (42327803, 42304121, 42574154, U2344218), the Open Fund of National Engineering Research Center of Oil and Gas Exploration Computer Software, and the Natural Science Foundation of Wuhan (2025040601020137).

References

[1] Tarantola A. Inversion of seismic reflection data in the acoustic approximation. Geophysics. 1984; 49(8):1259-1266. doi:10.1190/1.1441754

[2] Virieux J, Operto S. An overview of full-waveform inversion in exploration geophysics. Geophysics. 2009; 74(6): WCC1-WCC26. doi:10.1190/1.3238367

[3] Zhang X, Lomas A, Zhou M, Zheng Y, Curtis A. 3-D Bayesian variational full waveform inversion. Geophys J Int. 2023; 234(1):546-561. doi:10.1093/gji/ggad057

[4] Plessix RE. A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophys J Int. 2006; 167(2):495-503. doi:10.1111/j.1365-246X.2006.02978.x

[5] Wang T, Cheng J, Geng J. Reflection full waveform inversion with second-order optimization using the adjoint-state method. J Geophys Res Solid Earth. 2021; 126(8). doi:10.1029/2021jb022135

[6] Li YE, Demanet L. Full-waveform inversion with extrapolated low-frequency data. Geophysics. 2016; 81(6):R339-R348. doi:10.1190/geo2016-0038.1

[7] Métivier L, Allain A, Brossier R, Mérigot Q, Oudet E, Virieux J. Optimal transport for mitigating cycle skipping in full-waveform inversion: A graph-space transform approach. Geophysics. 2018; 83(5):R515-R540. doi:10.1190/geo2017-0807.1

[8] Teodor D, Comina C, Khosro Anjom F, Brossier R, Socco LV, Virieux J. Challenges in shallow target reconstruction by 3D elastic full-waveform inversion - which initial model? Geophysics. 2021; 86(4):R433-R446. doi:10.1190/geo2019-0523.1

[9] Li D, Xu K, Harris JM, Darve E. Coupled time-lapse full-waveform inversion for subsurface flow problems using intrusive automatic differentiation. Water Resour Res. 2020; 56(8):e2019WR027032. doi:10.1029/2019wr027032

[10] Zhu W, Xu K, Darve E, Beroza GC. A general approach to seismic inversion with automatic differentiation. Comput Geosci. 2021; 151:104751. doi:10.1016/j.cageo.2021.104751

[11] Adler A, Araya-Polo M, Poggio T. Deep learning for seismic inverse problems: Toward the acceleration of geophysical analysis workflows. IEEE Signal Process Mag. 2021; 38(2): 89-119. doi:10.1109/msp.2020.3037429

[12] Yu S, Ma J. Deep learning for geophysics: Current and future trends. Rev Geophys. 2021; 59(3):e2021RG000742. doi:10.1029/2021rg000742

[13] Yang F, Ma J. Deep-learning inversion: A next generation seismic velocity-model building method. Geophysics. 2019; 84(4):R583-R599. doi:10.1190/geo2018-0249.1

[14] Deng C, Feng S, Wang H, et al. OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion. Adv Neural Inf Process Syst. 2022; 35:6007-6020.

[15] Lewis W, Vigh D. Deep Learning Prior Models from Seismic Images for Full-Waveform Inversion. In: SEG International Exposition and Annual Meeting. SEG; 2017.

[16] Sun H, Demanet L. Extrapolated full-waveform inversion with deep learning. Geophysics. 2020; 85(3):R275-R288. doi:10.1190/geo2019-0195.1

[17] Zhang W, Gao J, Gao Z, Chen H. Adjoint-driven deep-learning seismic full-waveform inversion. IEEE Trans Geosci Remote Sens. 2020; 59(10):8913-8932. doi:10.1109/tgrs.2020.3044065

[18] Zhang ZD, Alkhalifah T. High-resolution reservoir characterization using deep learning-aided elastic full-waveform inversion: The North Sea field data example. Geophysics. 2020; 85(4):WA137-WA146. doi:10.1190/geo2019-0340.1

[19] Li Y, Alkhalifah T, Zhang Z. Deep-learning assisted regularized elastic full waveform inversion using the velocity distribution information from wells. Geophys J Int. 2021; 226(2):1322-1335. doi:10.1093/gji/ggab162

[20] Wang F, Huang X, Alkhalifah TA. A prior regularized full waveform inversion using generative diffusion models. IEEE Trans Geosci Remote Sens. 2023; 61:1-11. doi:10.1109/tgrs.2023.3337014

[21] Feng S, Lin Y, Wohlberg B. Multiscale data-driven seismic full-waveform inversion with field data study. IEEE Trans Geosci Remote Sens. 2021; 60:1-14. doi:10.1109/tgrs.2021.3114101

[22] Rasht-Behesht M, Huber C, Shukla K, Karniadakis GE. Physics-informed neural networks (PINNs) for wave propagation and full waveform inversions. J Geophys Res Solid Earth. 2022; 127(5):e25. doi:10.1029/2021jb023120

[23] Richardson A. Seismic Full-Waveform Inversion Using deep Learning Tools and Techniques. [arXiv Preprint]; 2018. doi:10.48550/arXiv.1801.07232

[24] Yang Y, Gao AF, Azizzadenesheli K, Clayton RW, Ross ZE. Rapid seismic waveform modeling and inversion with neural operators. IEEE Trans Geosci Remote Sens. 2023; 61:1-12. doi:10.1109/tgrs.2023.3264210

[25] Sun J, Niu Z, Innanen KA, Li J, Trad DO. A theory-guided deep-learning formulation and optimization of seismic waveform inversion. Geophysics. 2020; 85(2):R87-R99. doi:10.1190/geo2019-0138.1

[26] Song C, Wang Y, Richardson A, Liu C. Weighted envelope correlation-based waveform inversion using automatic differentiation. In: IEEE Transactions on Geoscience and Remote Sensing. Vol. 61. New York: IEEE; 2023. doi:10.1109/tgrs.2023.3300127

[27] Fang J, Zhou H, Elita Li Y, Shi Y, Li X, Wang E. Deep-learning optimization using the gradient of a custom objective function: A full-waveform inversion example study on the convolutional objective function. Geophysics. 2024; 89(5):R479-R492. doi:10.1190/geo2023-0538.1

[28] Zhang Y, Zhu X, Gao J. Seismic inversion based on acoustic wave equations using physics-informed neural network. IEEE Trans Geosci Remote Sens. 2023; 61:1-11. doi:10.1109/tgrs.2023.3236973

[29] Schuster GT, Chen Y, Feng S. Review of physics-informed machine-learning inversion of geophysical data. Geophysics. 2024; 89(6):1-91. doi:10.1190/geo2023-0615.1

[30] Li Z, Kovachki N, Azizzadenesheli K, et al. Fourier Neural Operator for Parametric Partial Differential Equations. [arXiv Preprint]; 2020. doi:10.48550/arxiv.2010.08895

[31] Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat Mach Intell. 2021; 3(3):218-229.

[32] Guo Z, Chai L, Huang S, Li Y. Inversion-DeepONet: A Novel DeepONet-Based Network with Encoder-Decoder for Full Waveform Inversion. [arXiv Preprint]; 2024. doi:10.48550/arXiv.2408.08005

[33] Ulyanov D, Vedaldi A, Lempitsky V. Deep Image Prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p. 9446-9454.

[34] Dittmer S, Kluth T, Maass P, Baguer DO. Regularization by architecture: A deep prior approach for inverse problems. J Math Imaging Vision. 2020; 62:456-470.

[35] Xie Y, Chen W, Ge H, Ng MK. Deep image prior and weighted anisotropic-isotropic total variation regularization for solving linear inverse problems. Appl Math Comput. 2024; 482:128952. doi:10.1016/j.amc.2024.128952

[36] Wu Y, McMechan GA. Parametric convolutional neural network-domain full-waveform inversion. Geophysics. 2019; 84(6):R881-R896. doi:10.1190/geo2018-0224.1

[37] He Q, Wang Y. Reparameterized full-waveform inversion using deep neural networks. Geophysics. 2021; 86(1): V1-V13. doi:10.1190/geo2019-0382.1

[38] Zhu W, Xu K, Darve E, Biondi B, Beroza GC. Integrating deep neural networks with full-waveform inversion: Reparametrization, regularization, and uncertainty quantification. Geophysics. 2022; 87(1):R93-R109. doi:10.1190/geo2020-0933.1

[39] Dhara A, Sen MK. Elastic full-waveform inversion using a physics-guided deep convolutional encoder-decoder. In: IEEE Transactions on Geoscience and Remote Sensing. New York: IEEE; 2023. doi:10.1109/tgrs.2023.3294427

[40] Sun J, Innanen K, Zhang T, Trad D. Implicit seismic full waveform inversion with deep neural representation. J Geophys Res Solid Earth. 2023; 128(3):e2022JB025964. doi:10.1029/2022jb025964

[41] Herrmann L, Bürchner T, Dietrich F, Kollmannsberger S. On the use of neural networks for full waveform inversion. Comput Methods Appl Mech Eng. 2023; 415:116278.doi:10.1016/j.cma.2023.116278

[42] Arridge S, Maass P, Öktem O, Schönlieb CB. Solving inverse problems using data-driven models. Acta Numerica. 2019; 28:1-174. doi:10.1017/s0962492919000059

[43] Sun P, Yang F, Liang H, Ma J. Full-waveform inversion using a learned regularization. In: IEEE Transactions on Geoscience and Remote Sensing. New York: IEEE; 2023. doi:10.1109/tgrs.2023.3322964

[44] Lord GJ, Powell CE, Shardlow T. An Introduction to Computational Stochastic PDEs. Vol. 50. Cambridge: Cambridge University Press; 2014.

[45] Benitez JAL, Furuya T, Faucher F, Tricoche X, Hoop M. Fine-Tuning Neural-Operator Architectures for Training and Generalization. [arXiv Preprint]; 2023.

[46] Ronneberger O, Fischer P, Brox T. U-net: Convolutional Networks for Biomedical Image Segmentation, in Medical Image Computing and Computer-Assisted Intervention- MICCAI 2015: 18^thInternational Conference, Munich, Germany, 2015, Proceedings, Part III 18. Berlin: Springer; 2015. p. 234-241.

[47] Siddique N, Paheding S, Elkin CP, Devabhaktuni V. U-net and its variants for medical image segmentation: A review of theory and applications. IEEE Access. 2021; 9:82031-82057.

[48] Baydin AG, Pearlmutter BA, Radul AA, Siskind JM. Automatic differentiation in machine learning: A survey. J Mach Learn Res. 2018; 18(153):1-43.

[49] Wang S, Jiang Y, Song P, Tan J, Liu Z, He B. Memory optimization in RNN-based full waveform inversion using boundary saving wavefield reconstruction. IEEE Trans Geosci Remote Sens. 2023; 61:1-12. doi:10.1109/tgrs.2023.3317529

[50] Yin Z, Orozco R, Louboutin M, Herrmann FJ. Solving multiphysics-based inverse problems with learned surrogates and constraints. Adv Model Simul Eng Sci. 2023; 10(1):14.

[51] Liu M, Vashisth D, Grana D, Mukerji T. Joint inversion of geophysical data for geologic carbon sequestration monitoring: A differentiable physics-informed neural network model. J Geophys Res Solid Earth. 2023; 128(3):e2022JB025372. doi:10.1029/2022jb025372