Fine-grained tree-species segmentation in UAV RGB imagery using parameter-efficient segment anything adaptation

Abstract

Accurate tree-species semantic segmentation from unmanned aerial vehicle (UAV) RGB imagery is essential for stand-level forest inventory, species composition assessment, and ecological management. Existing UAV RGB tree-species segmentation methods generally rely on discriminative image features, but they often struggle to preserve structural consistency under complex canopy conditions, such as shadow occlusion, scale variation, and subtle inter-species differences. As a result, they tend to produce fragmented crowns, noisy class boundaries, and unstable multi-class predictions. To address these issues, we propose HDP-SAM, a prompt-free and parameter-efficient SAM adaptation framework for UAV forest imagery. HDP-SAM leverages SAM’s pretrained mask priors and structural representations to enhance crown-level coherence. We further incorporate a dynamic prototype guidance (DPG) module into the decoder feature stream, where input-conditioned tree-species prototypes are derived from the current decoder features to recalibrate the dense feature stream before final classification, helping improve class assignment consistency and reduce confusion among visually similar species. We further adopt a hierarchical parameter-efficient fine-tuning (H-PEFT) strategy that combines adapters and LoRA across different stages of the SAM encoder to balance segmentation accuracy and parameter efficiency. Experiments on the Huangshan dataset and the Quebec Trees Zone 1 show that HDP-SAM consistently outperforms existing SAM adaptation methods and competitive remote-sensing baselines. It achieves 59.70% mIoU, 74.34% MF and 74.33% OA on Huangshan, and 79.92% mIoU, 88.40% MF and 91.66% OA on Quebec Trees Zone 1. Additional cross-zone evaluation on Quebec Trees Zones 2 and 3 further showed that HDP-SAM maintained leading performance under spatially separated testing. These results demonstrate the effectiveness of HDP-SAM for fine-grained tree-species segmentation in complex UAV forest scenes.

Keywords

Tree-species segmentation / UAV RGB remote sensing / Segment anything model (SAM) / Parameter-efficient fine-tuning

References

[1] Brockerhoff EG, Barbaro L, Castagneyrol B, Forrester DI, Gardiner B, González-Olabarria JR, Lyver POB, Meurisse N, Oxbrough A, Taki H, Thompson ID, van der Plas F, Jactel H. Forest biodiversity, ecosystem functioning and the provision of ecosystem services. Biodivers Conserv, 2017, 26(13): 3005-3035

[2] Buchner L, Eisen AK, Jochner-Oette S. How precise is precise enough? Tree crown segmentation using high resolution close-up multispectral UAV images and its effect on NDVI accuracy in Fraxinus excelsior L. trees. J for Res, 2025, 36(1 ArticleID: 137

[3] Chai S, Jain RK, Teng S, Liu J, Li Y, Tateyama T, Chen YW. Ladder fine-tuning approach for SAM integrating complementary network. Procedia Comput Sci, 2024, 246: 4951-4958

[4] Chen S, Ge C, Tong Z, Wang J, Song Y, Wang J, Luo P. AdaptFormer: adapting vision transformers for scalable visual recognition. Adv Neural Inf Process Syst, 2022, 35: 16664-16678

[5] Chen K, Liu C, Chen H, Zhang H, Li W, Zou Z, Shi Z. RSPrompter: learning to prompt for remote sensing instance segmentation based on visual foundation model. IEEE Trans Geosci Remote Sens, 2024, 62: 1-17

[6] Chen Y, Son M, Hua C, Kim JY. AoP-SAM: automation of prompts for efficient segmentation. Proc AAAI Conf Artif Intell, 2025, 39(2): 2284-2292

[7] Chen T, Zhu L, Ding C, Cao R, Wang Y, Zhang S, Li Z, Sun L, Zang Y, Mao P (2023) SAM-Adapter: adapting segment anything in underperformed scenes. In: 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp 3359–3367. https://doi.org/10.1109/ICCVW60793.2023.00361

[8] Cloutier M, Germain M, Laliberté E (2023) Quebec trees dataset. Zenodo. https://doi.org/10.5281/zenodo.8148479

[9] Cloutier M, Germain M, Laliberté E. Influence of temperate forest autumn leaf phenology on segmentation of tree species from UAV imagery using deep learning. Remote Sens Environ, 2024, 311 ArticleID: 114283

[10] Dainelli R, Toscano P, Di Gennaro SF, Matese A. Recent advances in unmanned aerial vehicle forest remote sensing—a systematic review. Part I: a general framework. Forests, 2021, 12(3 ArticleID: 327

[11] Dieste ÁG, Argüello F, Heras DB. ResBaGAN: a residual balancing GAN with data augmentation for forest mapping. IEEE J Sel Top Appl Earth Obs Remote Sens, 2023, 16: 6428-6447

[12] Ecke S, Dempewolf J, Frey J, Schwaller A, Endres E, Klemmt HJ, Tiede D, Seifert T. UAV-based forest health monitoring: a systematic review. Remote Sens, 2022, 14(13 ArticleID: 3205

[13] Fassnacht FE, White JC, Wulder MA, Næsset E. Remote sensing in forestry: current challenges, considerations and directions. Int J for Res, 2024, 97(1): 11-37

[14] Gamfeldt L, Snäll T, Bagchi R, Jonsson M, Gustafsson L, Kjellander P, Ruiz-Jaen MC, Fröberg M, Stendahl J, Philipson CD, Mikusiński G, Andersson E, Westerlund B, Andrén H, Moberg F, Moen J, Bengtsson J. Higher levels of multiple ecosystem services are found in forests with more tree species. Nat Commun, 2013, 4 ArticleID: 1340

[15] Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J (2017) A review on deep learning techniques applied to semantic segmentation. arXiv:1704.06857. https://arxiv.org/abs/1704.06857

[16] Guimarães N, Pádua L, Marques P, Silva N, Peres E, Sousa JJ. Forestry remote sensing from unmanned aerial vehicles: a review focusing on the data, processing and potentialities. Remote Sens, 2020, 12(6 ArticleID: 1046

[17] Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2022) LoRA: low-rank adaptation of large language models. In: International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=nZeVKeeFYf9

[18] Jadon S (2020) A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp 1−7. https://doi.org/10.1109/CIBCB48159.2020.9277638

[19] Kattenborn T, Leitloff J, Schiefer F, Hinz S. Review on convolutional neural networks (CNN) in vegetation remote sensing. ISPRS J Photogramm Remote Sens, 2021, 173: 24-49

[20] Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo WY, Dollár P, Girshick R (2023) Segment anything. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp 3992−4003. https://doi.org/10.1109/ICCV51070.2023.00371

[21] Kunwar S, Wang LQ, Kazempour Larsary M, Chaudhary R, Joshi PR, Ali A. Functional composition of tall-statured trees underpins aboveground biomass in tropical forests. J for Res, 2023, 34(2): 333-344

[22] Li C, Li K, Ji Y, Xu Z, Gu J, Jing W. A spatio-temporal multi-scale fusion algorithm for pine wood nematode disease tree detection. J for Res, 2024, 35(1): 109

[23] Lv J, Shen Q, Lv M, Li Y, Shi L, Zhang P. Deep learning-based semantic segmentation of remote sensing images: a review. Front Ecol Evol, 2023, 11 ArticleID: 1201125

[24] Ma X, Zhang X, Pun M-O. RS³Mamba: visual state space model for remote sensing image semantic segmentation. IEEE Geosci Remote Sens Lett, 2024, 21: 1-5

[25] Mäyrä J, Keski-Saari S, Kivinen S, Tanhuanpää T, Hurskainen P, Kullberg P, Poikolainen L, Viinikka A, Tuominen S, Kumpula T, Vihervaara P. Tree species classification from airborne hyperspectral and LiDAR data using 3D convolutional neural networks. Remote Sens Environ, 2021, 256 ArticleID: 112322

[26] McRoberts RE, Tomppo EO, Næsset E. Advances and emerging issues in national forest inventories. Scand J Forest Res, 2010, 25(4): 368-381

[27] Ni Z, Chen X, Zhai Y, Tang Y, Wang YLeonardis A, Ricci E, Roth S, Russakovsky O, Sattler T, Varol G. Context-guided spatial feature reconstruction for efficient semantic segmentation. Computer vision – ECCV 2024, 2025, Cham, Springer239-255

[28] Norouzi N, Orlova S, de Geus D, Dubbelman G (2024) ALGM: Adaptive local-then-global token merging for efficient semantic segmentation with plain vision transformers. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 15773−15782. https://doi.org/10.1109/CVPR52733.2024.01493

[29] Pu X, Jia H, Zheng L, Wang F, Xu F. ClassWise-SAM-Adapter: parameter-efficient fine-tuning adapts segment anything to SAR domain for semantic segmentation. IEEE J Sel Top Appl Earth Obs Remote Sens, 2025, 18: 4791-4804

[30] Quan Y, Li M, Hao Y, Liu J, Wang B. Tree species classification in a typical natural secondary forest using UAV-borne LiDAR and hyperspectral data. Gisci Remote Sens, 2023, 60(1 ArticleID: 2171706

[31] Schiefer F, Kattenborn T, Frick A, Frey J, Schall P, Koch B, Schmidtlein S. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks. ISPRS J Photogramm Remote Sens, 2020, 170: 205-215

[32] Shaharabany T, Dahan A, Giryes R, Wolf L (2023) AutoSAM: adapting SAM to medical images by overloading the prompt encoder. arXiv:2306.06370. https://arxiv.org/abs/2306.06370

[33] Sun Y, Jin X, Pukkala T, Li F. A hybrid method for tree-level forest planning. J for Res, 2025, 36(1): 62

[34] Wang L, Li R, Zhang C, Fang S, Duan C, Meng X, Atkinson PM. UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery. ISPRS J Photogramm Remote Sens, 2022, 190: 196-214

[35] Wang D, Zhang J, Du B, Xu M, Liu L, Tao D, Zhang L (2023) SAMRS: scaling-up remote sensing segmentation dataset with segment anything model. arXiv:2305.02034. https://doi.org/10.48550/arXiv.2305.02034

[36] Wu H, Huang P, Zhang M, Tang W, Yu X. CMTFNet: CNN and multiscale transformer fusion network for remote-sensing image semantic segmentation. IEEE Trans Geosci Remote Sens, 2023, 61: 1-12

[37] Yang Y, Yuan G, Li J. SFFNet: A wavelet-based spatial and frequency domain fusion network for remote sensing segmentation. IEEE Trans Geosci Remote Sens, 2024, 62: 1-17

[38] Zhang J, Li Y, Yang X, Jiang R, Zhang L. RSAM-Seg: a SAM-based model with prior knowledge integration for remote sensing image semantic segmentation. Remote Sens, 2025, 17(4): 590

[39] Zhang K, Liu D (2023) Customized segment anything model for medical image segmentation. arXiv:2304.13785. https://arxiv.org/abs/2304.13785

RIGHTS & PERMISSIONS

Northeast Forestry University