FragSAM: Near real-time rock fragment segmentation for size distribution analysis across diverse engineering domains

Abstract

Rock fragment size distribution (FSD) plays an important role in various engineering applications, such as mining, tunnelling, and other underground construction scenarios. While vision-based deep learning approaches have been increasingly applied to FSD analysis, they are often case-specific, showing limited cross-site generalization despite their accuracy. To address these challenges, FragSAM, an end-to-end, fully automated framework is proposed for near real-time rock fragment segmentation and FSD analysis across diverse engineering environments. FragSAM integrates the generalization power of Segment Anything Model (SAM) with a context-aware prompting mechanism and lightweight architecture for efficient dense fragment segmentation. In Stage 1, an enhanced SAM automatically generates high-quality annotations, which are used to train a modified CenterNet for precise centroid prediction. In Stage 2, these centroids serve as prompts for EdgeSAM, a lightweight SAM variant optimized for real-time inference. This two-stage design eliminates dense grid prompting and reduces reliance on heavy post-processing, enabling efficient and scalable segmentation. Experimental results show that FragSAM achieves competitive segmentation performance with significantly lower latency and model complexity compared to existing SAM-based methods. In comparison with supervised learning approaches, it also demonstrates superior generalization and performs better in low-quality or unseen scenarios. Furthermore, case studies on blasting fragmentation, TBM muck, and coastal rock surfaces confirm its robustness and seamless cross-site adaptability, requiring no tuning or retraining, making it highly practical for on-site applications.

Keywords

Segment anything model / Rock fragment size distribution / FragSAM / Prompt-based vision models / Near real-time segmentation

References

[1] Xiang ZZ, Yu ZX, Kang WH, Si GY, Oh J, Canbulat I. Estimation of in—situ rock strength from borehole geophysical logs in Australian coal mine sites. Int J Coal Geol 2023; 269:104210.

[2] Hu RZ, Wang G, Wang YH, Canbulat I, Si GY. From dynamic goaf formation to evolving spontaneous combustion and gas explosion hazard management. Process Saf Environ Prot 2025; 198:107131.

[3] Xiang ZZ, Kang WH, Ji YL, Si GY, Canbulat I, Lin HS, et al. Estimation of in—situ horizontal stresses based on multiscale borehole breakout data via machine learning: model development, validation and application. Geophys J Int 2025; 242:ggaf144.

[4] Wu XB, Dong JM, Hu RZ, Pang BX, Si GY. CFD modelling of prevention and mitigation of coal spontaneous combustion in longwall goaf: a comprehensive review and future outlook. Arch Comput Meth Eng 2025. in press.

[5] Al—Bakri AY, Sazid M. Application of artificial neural network (ANN) for prediction and optimization of blast—induced impacts. Mining 2021; 1(3):315-34.

[6] Miao YS, Zhang YP, Wu D, Li KB, Yan XR, Lin J. Rock fragmentation size distribution prediction and blasting parameter optimization based on the muck—pile model. Min Metall Explor 2021; 38(2):1071-80.

[7] Qiao WD, Zhao YF, Xu Y, Lei YM, Wang YJ, Yu S, et al. Deep learning—based pixel—level rock fragment recognition during tunnel excavation using instance segmentation model. Tunn Undergr Space Technol 2021; 115:104072.

[8] Tang YD, Wang YL, Si GY. Vision—based size distribution analysis of rock fragments using multi—modal deep learning and interactive annotation. Autom Constr 2024; 159:105276.

[9] Akbari M, Lashkaripour G, Yarahamdi Bafghi A, Ghafoori M. Blastability evaluation for rock mass fragmentation in Iran central iron ore mines. Int J Min Sci Technol 2015; 25(1):59-66.

[10] Lawal AI. A new modification to the Kuz—Ram model using the fragment size predicted by image analysis. Int J Rock Mech Min Sci 2021; 138:104595.

[11] Omotehinse AO, Taiwo BO. A comparative analysis on the performance of modified Kuz—Ram and Kuznetsov—Cunningham—Ouchterlony models on small and large diameter drill—hole blasts. Rock Mech Rock Eng 2023; 56(6):4251-69.

[12] Wang SF, Yin JJ, Pi ZZ, Cao WZ, Cai X, Zhou ZL. Automatic detection and characterization of discontinuity traces and rock fragment size distribution using a digital image processing method. Measurement 2024; 228:114343.

[13] Tang YD, He L, Lu W, Huang X, Wei H, Xiao HG. A novel approach for fracture skeleton extraction from rock surface images. Int J Rock Mech Min Sci 2021; 142:104732.

[14] Tang YD, He L, Xiao HG, Wang RH, Lu W, Xu T. Fracture extraction from smooth rock surfaces using depth image segmentation. Rock Mech Rock Eng 2021; 54(8):3873—89.

[15] Bamford T, Esmaeili K, Schoellig AP. A real—time analysis of post—blast rock fragmentation using UAV technology. Int J Min Reclam Environ 2017; 31(6):439-56.

[16] Mohammadi H, Barati B. Development of a rock fragmentation model for using in tunnel blasts. Geotech Geol Eng 2018; 36(4):2379—90.

[17] Moomivand H, Vandyousefi H. Development of a new empirical fragmentation model using rock mass properties, blasthole parameters, and powder factor. Arab J Geosci 2020; 13(22):1173.

[18] Azizi A, Moomivand H. A new approach to represent impact of discontinuity spacing and rock mass description on the median fragment size of blasted rocks using image analysis of rock mass. Rock Mech Rock Eng 2021; 54(4):2013—38.

[19] Agrawal A, Choudhary BS, Murthy VMSR. Seismic energy prediction to optimize rock fragmentation: a modified approach. Int J Environ Sci Technol 2022; 19(11):11301-22.

[20] Bamford T, Esmaeili K, Schoellig AP. A deep learning approach for rock fragmentation analysis. Int J Rock Mech Min Sci 2021; 145:104839.

[21] Fan HY, Tian ZH, Xu XB, Sun X, Ma YS, Liu HR, et al. Rockfill material segmentation and gradation calculation based on deep learning. Case Stud Constr Mater 2022; 17:e01216.

[22] Hu YK, Wang JJ, Wang XL, Yu J, Zhang J. Efficient virtual—to—real dataset synthesis for amodal instance segmentation of occlusion—aware rockfill material gradation detection. Expert Syst Appl 2024; 238:122046.

[23] Wang XL, Feng MG, Tang XX, Peng T, Li ZM, Yang CH. Ore image segmentation based on multiscale parallel efficient channel attention U—network. IFAC PapersOnLine 2024; 58(22):101-6.

[24] Zhou XX, Gong QM, Liu YQ, Yin LJ. Automatic segmentation of TBM muck images via a deep—learning approach to estimate the size and shape of rock chips. Autom Constr 2021; 126:103685.

[25] Liu Y, Si L, Wang ZB, Chen M, Li X, Wei D, et al. A novel coal—rock recognition method in coal mining face based on fusing laser point cloud and images. Int J Min Sci Technol 2025; 35(7):1057-71.

[26] Wang YL, Wang X, Tang YD, Dai X, Dong JM, Si GY. From laboratory to field: normal map—aided multimodal instance segmentation for blasting fragmentation analysis. Adv Eng Inf 2026; 71:104319.

[27] Huang GQ, Qin CJ, Wang HD, Liu CL. TBM rock fragmentation classification using an adaptive spot denoising and contour—texture decomposition attention—based method. Tunn Undergr Space Technol 2025; 161:106498.

[28] Li FL, Liu QS, Pan YC, Bo Y. An attention—enhanced ResNet model for classification of TBM rock chips. Tunn Undergr Space Technol 2025; 165:106916.

[29] Xie WQ, Zhang XP, Liu XL, Xu CY, Li XF, Song DQ, et al. Real—time perception of rock—machine interaction information in TBM tunnelling using muck image analysis. Tunn Undergr Space Technol 2023; 136:105096.

[30] Tang YD, Wang YL, Wang X, Oh J, Si GY. Automated scene—adaptive rock fragment recognition based on the enhanced segment anything model and fine—tuning RTMDet. Rock Mech Rock Eng 2025; 58(3):3973—99.

[31] Baek J, Choi Y. Image—based fragment size distribution analysis of muck pile using multiple spherical scales for improving accuracy and safety. Measurement 2025; 241:115776.

[32] Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, et al. Segment Anything. Paris: IEEE; 2023.p.3992-4003.

[33] Guo YP, Xu Y, Cui HT, Dang MH, Li SL. Segment anything model—based crack segmentation using low—rank adaption fine—tuning. Struct Health Monit 2025; 24(4):2579-91.

[34] Xu Y, Zhang CA, Li H. Transformer—based large vision model for universal structural damage segmentation. Autom Constr 2025; 176:106256.

[35] Wang JW, Zheng J, Hu J, Gong XJ, Lü Q, Han J, et al. An interactive framework integrating segment anything model and structure—from—motion for three—dimensional discontinuity identification in rock masses. Int J Min Sci Technol 2025; 35(10):1695-711.

[36] Zhao JJ, Li DY, Yu YS. Identification of rock fragments after blasting by using deep learning—based segment anything model. Minerals 2024; 14(7):654.

[37] Li F, Liu XY, Li ZP. A two—stage framework with ore—detect and segment anything model for ore particle segmentation and size measurement. IEEE Sens J 2025; 25(7):11722—36.

[38] Xiao YJ, Peng YQ, Wang M, Ning YF, Zhou YB, Kong KF, et al. A novel method for predicting coarse aggregate particle size distribution based on segment anything model and machine learning. Constr Build Mater 2024; 429:136429.

[39] Wang W, Yu C, Zhang TY, Chen FY, Liu YF, Liu YQ, et al. Oversized ore segmentation using SAM—enhanced U—Net with self—supervised pre—training and semi—supervised self—training. Expert Syst Appl 2025; 285:127980.

[40] Wang A, Chen H, Lin ZJ, Han JG, Ding GG. RepViT—SAM: Towards Real—Time Segmenting Anything 2024. https://doi.org/10.48550/arXiv.2312.05760.

[41] Xiong YY, Varadarajan B, Wu L, Xiang XY, Xiao FY, Zhu CC, Dai XL, Wang D, Sun F, Iandola F, Krishnamoorthi RK, Chandra V. EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything 2023. https://doi.org/10.48550/arXiv.2312.00863.

[42] Zhang C, Han D, Qiao Y, Kim JU, Bae S—H, Lee SK, Hong CS. Faster Segment Anything: Towards Lightweight SAM for Mobile Applications 2023. https://doi.org/10.48550/arXiv.2306.14289.

[43] Zhao X, Ding W, An Y, Du Y, Yu T, Li M, Tang MT, Wang JQ. Fast Segment Anything. arXiv preprint arXiv:2306.12156; 2023.

[44] Zhou C, Li XT, Loy CC, Dai B. EdgeSAM: Prompt—in—the—loop distillation for on—device deployment of SAM. Int J Comput Vis 2025; 133(12):8452-68.

[45] Zhou XY, Wang DQ, Krähenbühl P. Objects as Points 2019. https://doi.org/10.48550/arXiv.1904.07850.

[46] Duan KW, Bai S, Xie LX, Qi HG, Huang QM, CenterNet TQ. Keypoint Triplets for Object Detection 2019. https://doi.org/10.48550/arXiv.1904.08189.

[47] Lin T—Y, Maire M, Belongie S, Bourdev L, Girshick R, Hays J, et al. Context 2015. https://doi.org/10.48550/arXiv.1405.0312.

[48] He KM, Gkioxari G, Dollár P, Girshick R. Mask R—CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice: IEEE; 2017. p.2961—69.

[49] Wang XL, Zhang RF, Kong T, Li L, Shen CH. SOLOv2: Dynamic and Fast Instance Segmentation 2020. https://doi.org/10.48550/arXiv.2003.10152.

[50] Lyu CQ, Zhang WW, Huang HA, Zhou Y, Wang YD, Liu YY, Zhang SL, Chen K, et al. RTMDet: An Empirical Study of Designing Real—Time Object Detectors. arXiv preprint arXiv:2212.07784; 2022.