Development of a 3D-point cloud-based spatiotemporal series model for tunnel rock mass discontinuities prediction

Abstract

Predicting the three-dimensional (3D) distributions of discontinuities within rock masses is crucial for evaluating tunnel stability. However, this task is challenging due to the inherent opacity of rock, which prevents the direct observation of discontinuities. Most current methods for predicting discontinuities are based on extracting the two-dimensional intersection lines of spatial discontinuities. In this paper, we propose a novel, purely visual approach to analyze and predict the 3D distributions of discontinuities in rock masses. In this method, a 3D model of the tunnel face is constructed based on motion prediction and multi-view stereo vision, and the development of discontinuities is then predicted. Each set of discontinuities is projected onto the virtual tunnel face using a convex hull algorithm, creating a virtual trace. A newly developed algorithm for predicting spatiotemporal sequences, which incorporates a self-attention mechanism and a zigzag recurrent transition mechanism, is then applied to predict the evolution of discontinuities. For testing and verification, we used smartphones to collect surface data on multiple sets of excavated rock from the Bimoyuan Tunnel in Sichuan, China. Extensive experiments involving these surface data demonstrated the effectiveness of our proposed method. The findings provide technical support for predicting tunnel collapse and ensuring tunnel safety.

Keywords

Convolutional long short-term memory / Self-attention / Zigzag recurrent transition mechanism / Spatiotemporal series prediction / Tunnel rock mass discontinuities / 3D scene reconstruction

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

CRediT authorship contribution statement

Gang Yang: Writing - original draft, Visualization, Validation, Software, Resources, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Tianbin Li: Writing - review & editing, Supervision, Resources, Project administration, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This research was financially supported by the National Natural Science Foundation of China (Grant Nos. 42130719 and U19A20111) and the Sichuan Science and Technology Project (Grant No. 2021YFS0317).

We sincerely appreciate the invaluable support provided by the journal editors and the three anonymous reviewers. Their insightful comments and constructive suggestions have significantly improved the quality of this manuscript. We are truly grateful for their time, effort, and dedication.

Supplementary material

Supplementary data to this article can befoundonlineat https://doi.org/10.1016/j.undsp.2025.06.009.

References

[1] Baecher, G. B., Lanney, N. A., & Einstein, H. H. (1977). Statistical description of rock properties and sampling. Proceedings of the 18th U. S. Symposium on Rock Mechanics (USRMS).

[2] Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, B. D. (2009). PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics, 28 (3), 1-11.

[3] Barton, N., Lien, R., & Lunde, J. (1974). Engineering classification of rock masses for the design of tunnel support. Rock Mechanics, 6 (4), 189-236.

[4] Bradley, D., Boubekeur, T., & Heidrich, W. (2008). Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.1-8). Anchorage, AK, USA.

[5] Chen, J., Zhou, M., Huang, H., Zhang, D., & Peng, Z. (2021a). Automated extraction and evaluation of fracture trace maps from rock tunnel face images via deep learning. International Journal of Rock Mechanics and Mining Sciences, 142, 104745.

[6] Chen, N., Cai, X., Li, S., Zhang, X., & Jiang, Q. (2021b). Automatic extraction of rock mass discontinuity based on 3D laser scanning. Quarterly Journal of Engineering Geology and Hydrogeology, 54 (1), qjegh2020-054.

[7] Chen, N., Kemeny, J., Jiang, Q., & Pan, Z. (2017). Automatic extraction of blocks from 3D point clouds of fractured rock. Computers & Geosciences, 109, 149-161.

[8] Collins, R. T. (1996). A space-sweep approach to true multi-image matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.358-363). San Francisco, CA, USA.

[9] Dershowitz, W. S., & Einstein, H. H. (1988). Characterizing rock joint geometry with joint system models. Rock Mechanics and Rock Engineering, 21 (1), 21-51.

[10] Dong, X., Xu, Q., Huang, R., Liu, Q., & Kieffer, D. S. (2020). Reconstruction of surficial rock blocks by means of rock structure modelling of 3D TLS point clouds: The 2013 Long-Chang rockfall. Rock Mechanics and Rock Engineering, 53 (2), 671-689.

[11] Franklin, J. A., Maerz, N. H., & Bennett, C. P. (1988). Rock mass characterization using photoanalysis. International Journal of Mining and Geological Engineering, 6 (2), 97-112.

[12] Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (8), 1362-1376.

[13] Furukawa, Y., & Ponce, J. (2009). Carved visual hulls for image-based modeling. International Journal of Computer Vision, 81 (1), 53-67.

[14] Gao, Z., Tan, C., Wu, L., et al. (2022). SimVP: Simpler yet better video prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3160-3170). New Orleans, LA, USA.

[15] Giovanni, G., & Nicola, C. (2011). Semi-automatic extraction of rock mass structural data from high-resolution LiDAR point clouds. International Journal of Rock Mechanics and Mining Sciences, 48 (2), 187-198.

[16] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9 (8), 1735-1780.

[17] Hu, X., Huang, Z., Huang, A., Xu, J., & Zhou, S. (2023). A dynamic multi-scale voxel flow network for video prediction. arXiv preprint arXiv:2303.09875.

[18] Ju, Y., Wan, C., Fu, G., Ren, Z., Mao, L., & Chiang, F. P. (2022). Effects of near-fault stress evolution and surface asperities on rough fault slip: An evaluation based on photoelastic shear tests and additively printed models. Engineering Geology, 305, 106715.

[19] Kong, D., Wu, F., & Saroglou, C. (2020). Automatic identification and characterization of discontinuities in rock masses from 3D point clouds. Engineering Geology, 265, 105442.

[20] Kulatilake, P. (1993). Application of probability and statistics in joint network modeling in three dimensions. Probabilistic Methods in Geotechnical Engineering.

[21] Kulatilake, P., Wang, L., Tang, H., & Liang, Y. (2011). Evaluation of rock slope stability for Yujian River dam site by kinematic and block theory analyses. Computers and Geotechnics, 38 (6), 846-860.

[22] Le Guen, V., & Thome, N. (2020). Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.11471-11481). Seattle, WA, USA.

[23] Lhuillier, M., & Quan, L. (2005). A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (3), 418-433.

[24] Li, X., Chen, J., & Zhu, H. (2016). A new method for automated discontinuity trace mapping on rock mass 3D surface model. Computers & Geosciences, 89, 118-131.

[25] Li, Y., Tang, H., Zhu, F., & Hu, Z. (2023). An anisotropic elastoplastic strong discontinuity model for shear failure in anisotropic rock masses. Computers and Geotechnics, 176, 106762.

[26] Longoni, L., Arosio, D., Scaioni, M., Papini, M., Roncella, R., & Brambilla, D. (2012). Surface and subsurface non-invasive investigations to improve the characterization of a fractured rock mass. Journal of Geophysics and Engineering, 9 (5), 461-472.

[27] Matt, L., Mark, S., Diederichs, M., & Harrap, R. (2009). Optimization of LiDAR scanning and processing for automated structural evaluation of discontinuities in rock masses. International Journal of Rock Mechanics and Mining Sciences, 46 (1), 194-199.

[28] Mayer, J., Hamdi, P., & Stead, D. (2014). A modified discrete fracture network approach for geomechanical simulation. In Proceedings of the International Conference on Discrete Fracture Network Engineering. (pp.12). Vancouver, Canada.

[29] Okutomi, M., & Kanade, T. (1993). A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15 (4), 353-363.

[30] Otoo, J., Norbert, M., & Duan, Y. (2022). LiDAR and optical imaging for 3D fracture orientations. In Proceedings of the 2011 NSF Engineering Research and Innovation Conference. Atlanta, GA, USA.

[31] Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22 (10), 1345-1359.

[32] Paris, S., Sillion, F. X., & Quan, L. (2006). A surface reconstruction method using global graph cut optimization. International Journal of Computer Vision, 66 (2), 141-161.

[33] Pons, J. P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision, 72 (2), 179-193.

[34] Priest, S. D. (1985). Hemispherical projection methods in rock mechanics. International Journal of Rock Mechanics and Mining Sciences & Geomechanics Abstracts, 22 (5), 124.

[35] Riquelme, A., Abellán, A., Tomás, R., & Jaboyedoff, M. (2014). A new approach for semi-automatic rock mass joints recognition from 3D point clouds. Computers & Geosciences, 68, 38-52.

[36] Riquelme, A., Tomás, R., Cano, M., Pastor, J., & Abellán, A. (2018). Automatic mapping of discontinuity persistence on rock masses using 3D point clouds. Rock Mechanics and Rock Engineering, 51 (10), 3005-3028.

[37] Roncella, R., Forlani, G., & Remondino, F. (2005). Photogrammetry for geological applications: Automatic retrieval of discontinuity orientation in rock slopes. In In Proceedings of SPIE - The International Society for Optical Engineering (pp. 5665).

[38] Schönberger, J. L., Zheng, E., Frahm, J. M., & Pollefeys, M. (2016). Pixelwise view selection for unstructured multi-view stereo. In European conference on computer vision (pp. 501-518). Amsterdam, NL.

[39] Schönberger, J. L., & Frahm, J. M. (2016). Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.4104-4113). Las Vegas, NV, USA.

[40] Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.519-528). New York, NY, USA.

[41] Shi, X., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems, 28, 802-810.

[42] Slob, S., Hack, H., Feng, Q., Röshoff, K., & Turner, A.K. (2007). Fracture mapping using 3D laser scanning techniques. In Proceedings of the 11th Congress of the International Society for Rock Mechanics. (pp.299-302). Lisbon, PT.

[43] Strecha, C., Fransens, R., & Van Gool, L. (2006). Combined depth and outlier estimation in multi-view stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.2394-2401). New York, NY, USA.

[44] Tan, C., Gao, Z., Wu, L., Xu, Y., Xia, J., Li, S., & Li, S.Z. (2023). Temporal attention unit: Towards efficient spatiotemporal predictive learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 18770-18782). Vancouver, Canada.

[45] Tang, S., Li, C., Zhang, P., & Tang, R. (2023). SwinLSTM: Improving spatiotemporal prediction accuracy using Swin Transformer and LSTM. In IEEE/CVF International Conference on Computer Vision. (pp. 13470-13479). Paris, France.

[46] Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, W. A. (2000). Bundle adjustment: A modern synthesis. In Lecture Notes in Computer Science, 1883 (1), 153-177.

[47] Ulusay, R. (2015). The ISRM suggested methods for rock characterization, testing and monitoring: 2007-2014. Springer.

[48] Wang, Y., Long, M., Wang, J., Gao, Z., & Yu, P. S. (2017). PredRNN: Recurrent neural networks for predictive learning using spatiotemporal LSTMs. In Advances in neural information processing systems. (pp.879-888). Long Beach, CA, USA.

[49] Wang, Y., Zhang, J., & Zhu, H. (2019). Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9146-9154). Long Beach, CA, USA.

[50] Wang, X., & Cai, M. (2020). A DFN-DEM multi-scale modeling approach for simulating tunnel excavation response in jointed rock masses. Rock Mechanics and Rock Engineering, 53 (3), 1053-1077.

[51] Warburton, P. M. (1980). A stereological interpretation of joint trace data. International Journal of Rock Mechanics and Mining Sciences & Geomechanics Abstracts, 17 (4), 181-190.

[52] Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3 (1), 9.

[53] Xue, Y., Cao, Y., Zhou, M., Zhang, F., Shen, K., & Jia, F. (2023). Rock mass fracture maps prediction based on spatiotemporal image sequence modeling. Computer-Aided Civil and Infrastructure Engineering, 38 (4), 470-488.

[54] Yan, J., Chen, J., Zhang, Y., Liu, Y., Zhao, X., Xue, J., Zhu, C., Mehmood, Q., & Wang, Q. (2023). Semi-automatic extraction of dangerous rock blocks from jointed rock exposures based on a discontinuity trace map. Computers and Geotechnics, 156, 105265.

[55] Yang, G., Li, T. B., Tang, H., Xing, D., Hu, Y., & Li, S. (2024). Evaluation method for the intactness of the tunnel face surrounding rock based on tunnel face images. Quarterly Journal of Engineering Geology and Hydrogeology, 57 (3), qjegh2024-018.

[56] Yao, Y., Luo, Z., Li, S., et al. (2019). Recurrent MVSNet for high-resolution multi-view stereo depth inference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp.5520-5529). Long Beach, CA, USA.

[57] Zaharescu, A., Boyer, E., & Horaud, R. (2007). TransforMesh:A topology-adaptive mesh-based approach to surface evolution. In Asian Conference on Computer Vision (pp. 166-175). China: Hong Kong.

[58] Zhan, J., Chen, J., Xu, P., Han, X., Chen, Y., Ruan, Y., & Zhou, X. (2017). Computational framework for obtaining volumetric fracture intensity from 3D fracture network models using Delaunay triangulations. Computers and Geotechnics, 89, 179-194.

[59] Zhang, L., & Einstein, H. H. (2000). Estimating the intensity of rock discontinuities. International Journal of Rock Mechanics and Mining Sciences, 37 (5), 819-837.

[60] Zhang, Q., Wang, X., Zhu, H., Zhang, K., & Li, X. (2022). Mixture distribution model for three-dimensional geometric attributes of multiple discontinuity sets based on trace data of rock mass. Engineering Geology, 311, 106915.

[61] Zhong, Y., Liang, L., & Zharkov, I. (2023). MMVP: Motion-matrix-based video prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4250-4260). Vancouver, Canada.

[62] Zhu, H., Zuo, Y., Li, X., Deng, J., & Zhuang, X. (2014). Estimation of the fracture diameter distributions using the maximum entropy principle. International Journal of Rock Mechanics and Mining Sciences, 72, 127-137.

[63] Zhu, Q., Min, C., Wei, Z., Wei, Z., Chen, Y., & Wang, G. (2021). Deep learning for multi-view stereo via plane sweep: A survey. arXiv preprint arXiv:2106.15328.