3D-S-wave-LLM: three-dimensional shear wave velocity forecasting by large language models

Abstract

This study addresses the issue of spatial coverage blind spots induced by the limited three-dimensional (3D) shear wave (S-wave) velocity resolution of conventional surface wave imaging methods. To resolve this problem, a 3D S-wave velocity forecasting method based on large language models (LLMs) is proposed. Specifically, we design a prompting template tailored to the 3D S-wave velocity forecasting task combined with a 3D S-wave velocity field grid traversal strategy. The approach converts previously inverted 3D S-wave velocity data into a 3D S-wave velocity prompt sequence dataset with spatiotemporal sequence characteristics. Then, by fine-tuning the LLM using a standard trainer, the fine-tuned LLM is leveraged to forecast S-wave velocities in areas of the study region where coverage is incomplete. Experimental results demonstrate that LLM models ( after fine-tuning via prompting techniques and optimization through data engineering) exhibit a good forecasting performance and generalization ability in the scenario of 3D S-wave velocity prediction. These results validate the feasibility and reliability of the proposed method. The findings of this study can be applied to high-resolution S-wave velocity imaging and provide a valuable reference for in-depth investigations of complex geological structures within the geoscience field.

Keywords

three-dimensional (3D) shear wave (S-wave) velocity / large language models (LLMs) / shear wave (S-wave) velocity forecasting

References

[1] WANG H F, MA J F, LIN L, et al. Research on predicting S-wave velocity in tight reservoir: Sulige gas field for example. Progress in Geophysics, 2019, 34(4): 1521-1529. (in Chinese)

[2] OLUTOKI J O, ZHAO J G, SIDDIQUI N A, et al. Shear wave velocity prediction: A review of recent progress and future opportunities. Energy Geoscience, 2024, 5(4): Article 100338.

[3] BENSEN G D, RITZWOLLER M H, BARMIN M P, et al. Processing seismic ambient noise data to obtain reliable broad-band surface wave dispersion measurements. Geophysical Journal International, 2007, 169(3): 1239-1260.

[4] CAMPILLO M, PAUL A . Long-range correlations in the diffuse seismic coda. Science, 2003, 299: 547-549.

[5] ROUX P, SABRA K G, KUPERMAN W A, et al. Ambient noise cross correlation in free space: Theoretical approach. The Journal of the Acoustical Society of America, 2005, 117(1): 79-84.

[6] WEAVER R. Information from seismic noise. Science, 2005, 307: 1568-1569.

[7] SHAPIRO N, CAMPILLO M, STEHLY L, et al. High-resolution surface-wave tomography from ambient seismic noise. Science, 2005, 307: 1615-1618.

[8] YAO H J, VAN DER HILST R D, DE HOOP M V . Surface-wave array tomography in SE Tibet from ambient seismic noise and two-station analysis-i. Phase velocity maps. Geophysical Journal International, 2006, 166(2): 732-744.

[9] TAN Y H, YU X W, ZHANG W B . Three-dimensional shear wave velocity structure of the crust and uppermost mantle in the northeastern margin of the Tibetan plateau and surrounding areas based on the ambient noise. Chinese Journal of Geophysics, 2025, 68(2): 499-516. (in Chinese)

[10] CHEN Y, BAO Q Z, HU J, et al. Analysis of 3-D velocity structure in Hanzhong basin and adjacent areas based on ambient noise tomography from dense seismic arrays. Chinese Journal of Geophysics, 2025, 68(8): 3103-3118. (in Chinese)

[11] LI M Y, ZENG X W, YAO H J, et al. Three-dimensional crustal velocity structure and tectonic implication of the Haiyuan-Liupan Mountain Area based on ambient noise tomography. Chinese Journal of Geophysics, 2024, 67(12): 4666-4680. (in Chinese)

[12] ZHAO X B, IRVIN L, GALETTI E, et al. Bayesian surface wave inversion for 3D shear wave velocity structure beneath the British Isles: Comparing direct-3D variational inversion to two-step (2D + 1D) inversion methods. Preprint, arXiv: 2507.15390, 2025.

[13] HOUSER P I Q, PENNINGTON D D, KUBICKI J D, et al. Enhancing basic geology skills with artificial intelligence: An exploration of automated reasoning in field geology. https://api.semanticscholar.org/CorpusID:263166549. 2023.

[14] GHYSELINCKS S, OKHMAK V, ZAMPINI S, et al. Synthetic geology: Structural geology meets deep learning. Journal of Geophysical Research: Machine Learning and Computation, 2026, 3(1): 20p.

[15] WANG H, CHEN Y F, GU Y J, et al. Multidimensional reconstruction of noise correlation functions and its application in improving surface-wave inversion. Geophysical Journal International, 2025, 242(1): doi: 10.1093/gji/ggaf159.

[16] ZHANG F B, HAN S, ZENG A P . 3D shear wave velocity inversion with passive seismic surface wave in a mineral exploration. Progress in Geophysics, 2024, 39(5): 2013-2024. (in Chinese)

[17] NAKAMURA Y, SHIBUTANI T . Three-dimensional shear wave velocity structure in the upper mantle beneath the Philippine Sea region. Earth, Planets and Space, 1998, 50: 939-952.

[18] MOSCHETTI M, RITZWOLLER M, LIN F C, et al. Crustal shear wave velocity structure of the western United States inferred from ambient seismic noise and earthquake data. Journal of Geophysical Research, 2010, 115: doi: 10.1029/2010JB007448.

[19] RY R V, CUMMINS P R, HEJRANI B, et al. 3D shallow shear velocity structure of the Jakarta Basin from transdimensional ambient noise tomography. Geophysical Journal International, 2023, 234(3): 1916-1932.

[20] SOUZA J L D, SANTOS N P D, VILAR C D S . Three-dimensional shear wave velocity structure of the northeastern Brazilian lithosphere. International Journal of Geosciences, 2016, 7: 849-872.

[21] LEE S H H . Regression models of shear wave velocities in Taipei Basin. Journal of the Chinese Institute of Engineers, 1990, 13(5): 519-532.

[22] AKHUNDI H, GHAFOORI M, LASHKARIPOUR G R. Prediction of shear wave velocity using artificial neural network technique, multiple regression and petrophysical data: A case study in Asmari reservoir (SW Iran). Open Journal of Geology, 2014, 4(7): 303-313.

[23] MOHAMMAD M, RAMEZANZADEH A, TOKHMECHI B . Shear wave travel time estimation from petrophysical logs using ANFIS-PSO algorithm: A case study from Ab-Teymour Oilfield. Journal of Natural Gas Science and Engineering, 2017, 38: 373-387.

[24] QI Q M, MüLLER T, PERVUKHINA M . Sonic QP/QS ratio as diagnostic tool for shale gas saturation. Geophysics, 2017, 82(3): MR97-MR103.

[25] KRIEF M, GARAT J. STELLING-WERFF J, et al. A petrophysical interpretation using the velocities of P and S waves (full-waveform sonic). Petrophysics, 1990, 31(6): 355-369.

[26] XU S Y, WHITE R E . A new velocity model for clay-sand mixture. Geophysical Prospecting, 1995, 43(1): 91-118.

[27] AZADPOUR M, SABERI M R, JAVAHERIAN A, et al. Rock physics model-based prediction of shear wave velocity utilizing machine learning technique for a carbonate reservoir, southwest Iran. Journal of Petroleum Science and Engineering, 2020, 195: Article 107864.

[28] MALEKI S, MORADZADEH A, GHAVAMI R, et al. Prediction of shear wave velocity using empirical correlations and artificial intelligence methods. NRIAG Journal of Astronomy and Geophysics, 2014, 3(1): 70-81.

[29] MOHAMMAD M, RAMEZANZADEH A, AMIRI H, et al. Machine learning technique for the prediction of shear wave velocity using petrophysical logs. Journal of Petroleum Science and Engineering, 2019, 174: 306-327.

[30] ASOODEH M, BAGHERIPOUR P . Prediction of compressional, shear, and Stoneley wave velocities from conventional well log data using a committee machine with intelligent systems. Rock Mechanics and Rock Engineering, 2012, 45(1): 45-63.

[31] COVA D, LIU Y . Shear wave velocity prediction using bidirectional recurrent gated graph convolutional network with total information embeddings. Frontiers in Earth Science, 2023, 11: Article 1101601.

[32] FU X G, WEI Y H, SU Y, et al. Shear wave velocity prediction using long short-term memory network with generative adversarial mechanism. PLOS One, 2025, 20(6): Article e0325271.

[33] AKINYEMI O D, ELSAADANY M, SIDDIQUI N A, et al. Machine learning application for prediction of sonic wave transit time: A case of Niger Delta basin. Results in Engineering, 2023, 20: Article 101528.

[34] FANG H J, YAO H J, ZHANG H J, et al. Direct inversion of surface wave dispersion for three-dimensional shallow crustal structure based on ray tracing: Methodology and application. Geophysical Journal International, 2015, 201(3): 1251-1263.

[35] GRUVER N, FINZI M, QIU S K, et al. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems 36: Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS'23), 2023, Dec 10-16, New Orleans, LA, USA. Red Hook, NY, USA: Curran Associates, Inc, 2023: 19622-19635.

[36] JIN M, ZHANG Y F, CHEN W, et al. Position: What can large language models tell us about time series analysis. Proceedings of the 41st International Conference on Machine Learning (ICML'24), 2024, Jul 21-27, Vienna, Austria. 2024: 22260-22276.

[37] LIU C X, XU Q X, MIAO H, et al. TimeCMA: Towards LLM-empowered multivariate time series forecasting via cross-modality alignment. Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI'25), 37th Conference on Innovative Applications of Artificial Intelligence (IAAI'25), and 15th Symposium on Educational Advances in Artificial Intelligence (EAAI'25): Vol 18(AAAI Technical Tracks), 2025, Feb 25-Mar 4, Philadelphia, PA, USA. Red Hook, NY, USA: Curran Associates, Inc, 2024: 18780-18788.

[38] XUE H, SALIM F D. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(11): 6851-6864.

[39] JIN M, WANG S Y, MA L T, et al. Time-LLM: Time series forecasting by reprogramming large language models. arXiv Preprint, arXiv:2310.01728, 2024.

[40] LIU X, HU J F, LI Y, et al. UniTime: A language-empowered unified model for cross-domain time series forecasting. Proceedings of the 2024 ACM on Web Conference (WWW '24), 2024, May 13-17, Singapore. New York, NY, USA: ACM, 2024: 4095-4106.

[41] PAN Z J, JIANG Y S, GARG S, et al. S2IP-LLM: Semantic space informed prompt learning with LLM for time series forecasting. Proceedings of the 41st International Conference on Machine Learning (ICML'24), 2024, Jul 21-27, Vienna, Austria. 2024: 39135-3915.

[42] SUN C X, LI Y L, LI H Y, et al. TEST: Text prototype aligned embedding to activate LLM's ability for time series. arXiv Preprint, arXiv:2308.08241, 2023.

[43] HU J X, HU Y H, CHEN W, et al. Attractor memory for long-term time series forecasting: A chaos perspective. Advances in Neural Information Processing Systems 37: Proceedings of the 38th International Conference on Neural Information Processing Systems (NeurIPS'24), 2024, Dec 10-15, Vancouver, Canada. Red Hook, NY, USA: Curran Associates, Inc, 2024: 20786-20818.

[44] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Advances in Neural Information Processing Systems 30: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 998-6008.

[45] ZHOU H Y, ZHANG S H, PENG J Q, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI'21), 2021, Feb 2-9, Online. Red Hook, NY, USA: Curran Associates Inc, 2021: 11106-11115.

[46] WU H X, XU J H, WANG J M, et al. Autoformer: Decomposition transformers with autocorrelation for long-term series forecasting. Advances in Neural Information Processing Systems 34: Proceedings of the 35th Annual Conference on Neural Information Processing System (NeurIPS'21), 2021, Dec 6-14, virtual. 2021: 22419-22430.

[47] DEVLIN J, CHANG M W, KENTON LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT'19): Vol 1(Long and Short Papers), 2019, Jun 2-7, Minneapolis, MN, USA. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 4171-4186.

[48] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS'20), 2020, Dec 6-12, virtual. Red Hook, NY, USA: Curran Associates Inc, 2020: 1877-1901.

[49] CLARK K, LUONG M T, LE Q, et al. Electra: Pre-training text encoders as discriminators rather than generators. arXiv Preprint, arXiv: 2003.10555, 2020.

[50] LIU Y H, OTT M, GOYAL N, et al. RoBERTa: A robustly optimized BERT pretraining approach. arXiv Preprint, arXiv: 1907.11692, 2019.

[51] LEWIS M, LIU Y H, GOYAL N, et al. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, Jul 5-10, Online. Stroudsburg, PA, USA: Association for Computational Linguistics, 2020: 7871-7880.

[52] BELTAGY I, PETERS M, COHAN A . Longformer: The long-document transformer. arXiv Preprint, arXiv: 2004.05150, 2020.

[53] RAFFEL C, SHAZEER N, ROBERTS A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21(140): 1-67.