Solving Traveltime Tomography with Deep Learning

Abstract

This paper introduces a neural network approach for solving two-dimensional traveltime tomography (TT) problems based on the eikonal equation. The mathematical problem of TT is to recover the slowness field of a medium based on the boundary measurement of the traveltimes of waves going through the medium. This inverse map is high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this and filtered back-projection, we propose an effective neural network architecture for the inverse map using the recently proposed BCR-Net, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks.

Keywords

Traveltime tomography / Eikonal equation / Inverse problem / Neural networks / Convolutional neural network

References

[1] Abadi, M. et al.: Tensorflow: a system for large-scale machine learning., Osdi, 16, 265–283, (2016)

[2] Adler J, Öktem O. Solving ill-posed inverse problems using iterative deep neural networks. Inverse Probl.. 2017, 33 12

[3] Antun V, Renna F, Poon C, Adcock B, Hansen AC. On instabilities of deep learning in image reconstruction and the potential costs of AI. Proceed. Nat. Acad. Sci.. 2020, 117 48 30088-30095

[4] Araya-Polo M, Jennings J, Adler A, Dahlke T. Deep-learning tomography. Lead. Edge. 2018, 37 1 58-66

[5] Backus G, Gilbert F. The resolving power of gross Earth data. Geophys. J. Int.. 1968, 16 2 169-205

[6] Bar, L., Sochen, N.: Unsupervised deep learning algorithm for PDE-based forward and inverse problems. arXiv preprint. (2019). arXiv:1904.05417

[7] Berg J, Nyström K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing. 2018, 317 28-41

[8] Beylkin G, Coifman R, Rokhlin V. Fast wavelet transforms and numerical algorithms I. Commun. Pure Appl. Math.. 1991, 44 2 141-183

[9] Born, M., Wolf, E.: Principles of optics: electromagnetic theory of propagation, interference and diffraction of light, 3, Oxford: Pergamon, (1965)

[10] Carleo G, Troyer M. Solving the quantum many-body problem with artificial neural networks. Science. 2017, 355 6325 602-606

[11] Chung E, Qian J, Uhlmann G, Zhao H. An adaptive phase space method with application to reflection traveltime tomography. Inverse Probl.. 2011, 27 11

[12] Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. In: International conference on learning representations. https://openreview.net/forum?id=Hkbd5xZRb, (2018)

[13] Colbrook, M.J., Antun, V., Hansen, A.C.: Can stable and accurate neural networks be computed?–on the barriers of deep learning and smale’s 18th problem. arXiv preprint. (2021). arXiv:2101.08286

[14] Cybenko G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst.. 1989, 2 4 303-314

[15] de Hoop, M.V., Lassas, M., Wong, C.A.: Deep learning architectures for nonlinear operator functions and nonlinear inverse problems. arXiv preprint. (2019). arXiv:1912.11090

[16] Deckelnick K, Elliott CM, Styles V. Numerical analysis of an inverse problem for the eikonal equation. Numerische Mathematik. 2011, 119 2 245

[17] Dozat, T.: Incorporating Nesterov momentum into adam. In: International Conference on Learning Representations. (2016)

[18] E Weinan YB. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat.. 2018, 6 1 1-12

[19] Fan Y, Bohorquez CO, Ying L. BCR-Net: a neural network based on the nonstandard wavelet form. J. Comput. Phys.. 2019, 384 1-15

[20] Fan Y, Feliu-Fabà J, Lin L, Ying L, Zepeda-Núñez L. A multiscale neural network based on hierarchical nested bases. Res. Math. Sci.. 2019, 6 2 21

[21] Fan, Yuwei., Lin, Lin., Ying, Lexing., Zepeda-Núñez, Leonardo.: A multiscale neural network based on hierarchical matrices. arXiv preprint. (2018). arXiv:1807.01883

[22] Fan, Y., Ying, L.: Solving electrical impedance tomography with deep learning. arXiv preprint. (2019). arXiv:1906.03944

[23] Fan, Y., Ying, L.: Solving optical tomography with deep learning. arXiv preprint. (2019). arXiv:1910.04756

[24] Feliu-Faba, J., Fan, Y., Ying, L.: Meta-learning pseudo-differential operators with deep neural networks. arXiv preprint. (2019). arXiv:1906.06782

[25] Genzel, M., Macdonald, J., März, M.: Solving inverse problems with deep neural networks–robustness included?. arXiv preprint. (2020). arXiv:2011.04268

[26] Gilton D, Ongie G, Willett R. Neumann networks for linear inverse problems in imaging. IEEE Trans. Comput. Imag.. 2019, 6 328-343

[27] Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. 249–256. (2010)

[28] Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep learning, MIT press Cambridge, 1. (2016)

[29] Gottschling, N.M., Antun, V., Adcock, B., Hansen, A.C.: The troublesome kernel: why deep learning for inverse problems is typically unstable. arXiv preprint. (2020). arXiv:2001.01258

[30] Han J, Jentzen A, E W. Solving high-dimensional partial differential equations using deep learning. Proceed. Nat. Academy Sci.. 2018, 115 34 8505-8510

[31] Han J, Zhang L, Car R, E W. Deep potential: a general representation of a many-body potential energy surface. Commun. Comput. Phys.. 2018, 23 3 629-639

[32] Hauptmann A, Adler J, Arridge S, Öktem O. Multi-scale learned iterative reconstruction. IEEE Trans. Comput. Imag.. 2020, 6 843-856

[33] Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag.. 2012, 29 6 82-97

[34] Hoole S, Ratnajeevan H. Artificial neural networks in the solution of inverse electromagnetic field problems. IEEE Trans. Magn.. 1993, 29 2 1931-1934

[35] Ishii, H.: A simple, direct proof of uniqueness for solutions of the Hamilton-Jacobi equations of eikonal type. Proceed. Am. Math. Soc. (1987)

[36] Jin X, Wang LV. Thermoacoustic tomography with correction for acoustic speed variations. Phys. Med. Biol.. 2006, 51 24 6437

[37] Kabir H, Wang Y, Yu M, Zhang QJ. Neural network inverse modeling and applications to microwave filter design. IEEE Trans. Microw. Theory Tech.. 2008, 56 4 867-879

[38] Kao Chiu-Yen, Osher S, Tsai YH. Fast sweeping methods for static Hamilton-Jacobi equations. SIAM J. Numer. Anal.. 2005, 42 6 2612-2632

[39] Khoo, Y., Lu, J., Ying, L.: Solving parametric PDE problems with artificial neural networks. arXiv preprint. (2017). arXiv:1707.03351

[40] Khoo Y, Lu J, Ying L. Solving for high-dimensional committor functions using artificial neural networks. Res Math Sci. 2019, 6 1 1

[41] Khoo, Y., Ying, L.: SwitchNet: a neural network model for forward and inverse scattering problems. arXiv preprint. (2018). arXiv:1810.09675

[42] Kosovichev AG. Tomographic imaging of the Sun’s interior. Astrophys. J. Lett.. 1996, 461 1 L55

[43] Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems - vol. 1, NIPS’12, Curran Associates Inc., USA, 1097–1105, (2012)

[44] Kutyniok, G., Petersen, P., Raslan, M., Schneider, R.: A theoretical analysis of deep neural networks and parametric PDEs. arXiv preprint. (2019). arXiv:1904.00377

[45] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015, 521 436

[46] Leung MKK, Xiong HY, Lee LJ, Frey BJ. Deep learning of the tissue-regulated splicing code. Bioinformatics. 2014, 30 12 i121-i129

[47] Leung S, Qian J. An adjoint state method for three-dimensional transmission traveltime tomography using first-arrivals. Commun. Math. Sci.. 2006, 4 1 249-266

[48] Li, Y., Lu, J., Mao, A.: Variational training of neural network approximations of solution maps for physical models. arXiv preprint. (2019). arXiv:1905.02789

[49] Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-net: Learning PDEs from data. In: Proceedings of the 35th International Conference on Machine Learning, Dy, Jennifer, Krause, Andreas, Proceedings of Machine Learning Research, 80, PMLR, Stockholmsmässan, Stockholm Sweden. pp 3208–3216 (2018). http://proceedings.mlr.press/v80/long18a.html

[50] Lucas A, Iliadis M, Molina R, Katsaggelos AK. Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEE Signal Process. Mag.. 2018, 35 1 20-36

[51] Ma J, Sheridan RP, Liaw A, Dahl GE, Svetnik V. Deep neural nets as a method for quantitative structure-activity relationships. J. Chem. Inf. Model.. 2015, 55 2 263-274

[52] Monga V, Li Y, Eldar YC. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing. IEEE Signal Process. Mag.. 2021, 38 2 18-44

[53] Munk, W., Worcester, P., Wunsch, C.: Ocean acoustic tomography, Cambridge university press, (2009)

[54] Popovici, A.M., Sethian, J.: Three dimensional traveltime computation using the fast marching method, Seg technical program expanded abstracts 1997. Society of Exploration Geophysicists. 1778–1781 (1997)

[55] Putzky Patrick. Welling, Max, Invert to learn to invert. Adv. Neural Inf. Process. Syst.. 2019, 32 446-456

[56] Qian J, Zhang YT, Zhao HK. Fast sweeping methods for eikonal equations on triangular meshes. SIAM J. Numer. Anal.. 2007, 45 1 83-107

[57] Raissi M, Karniadakis GE. Hidden physics models: machine learning of nonlinear partial differential equations. J. Comput. Phys.. 2018, 357 125-141

[58] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys.. 2019, 378 686-707

[59] Rawlinson N, Pozgay S, Fishwick S. Seismic tomography: a window into deep Earth. Phys. Earth Planet. Inter.. 2010, 178 3–4 101-135

[60] Rudd K, Ferrari S. A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks. Neurocomputing. 2015, 155 277-285

[61] Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw.. 2015, 61 85-117

[62] Schomberg H. An improved approach to reconstructive ultrasound tomography. J. Phys. D Appl. Phys.. 1978, 11 15 L181

[63] Schuster GT, Quintus-Bosz A. Wavepath eikonal traveltime inversion: theory. Geophysics. 1993, 58 9 1314-1323

[64] Sethian JA. Fast marching methods. SIAM Rev.. 1999, 41 2 199-235

[65] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27, Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q., Curran Associates, Inc., 3104–3112, (2014)

[66] Tan C, Lv S, Dong F, Takei M. Image reconstruction based on convolutional neural network for electrical resistance tomography. IEEE Sens. J.. 2018, 19 1 196-204

[67] Üstündag, D.: Retrieving slowness distribution of a medium between two boreholes from first arrival traveltimes. Int. J. Geol 2, 1–8, (2008)

[68] Yeung, T.S.A., Chung, E.T., Uhlmann, G.: Numerical inversion of 3d geodesic X-ray transform arising from traveltime tomography. arXiv preprint. (2018). arXiv:1804.10006

[69] Zhang HM, Dong B. A review on deep learning in medical image reconstruction. J. Oper. Res. Soc. China. 2020

[70] Zhao H. A fast sweeping method for eikonal equations. Math. Comput.. 2005, 74 250 603-627

Funding

National Science Foundation(DMS-1818449)