MTSCFNet: a novel framework for improving tree species classification in a subtropical forest using RGB, LiDAR-derived, and GF-2 data

Abstract

Accurate individual tree species classification is essential for forest inventory, management, and conservation. However, existing methods relying primarily on single-source remote sensing data (e.g., spectral, LiDAR, or RGB) often suffer from insufficient feature representation and noise interference, particularly in subtropical forests with high species diversity, leading to increased classification errors. To address these challenges, we proposed the Multi-source Tree Species Classification Fusion Network (MTSCFNet), a novel deep learning framework that integrates RGB imagery, LiDAR-derived feature maps, and GF-2 satellite data through a modified UNet backbone, which incorporates a three-branch encoder and a Triple Branch Feature Fusion (TBFF) module within a middle fusion strategy. We evaluated the MTSCFNet in Chinese-fir mixed forests located in the Shanxia Forest Farm, Jiangxi Province, China. The results showed that: (1) MTSCFNet outperformed four baseline models, achieving Macro F1 (0.78 ± 0.01), Micro F1 (0.93 ± 0.01), Weighted F1 (0.93 ± 0.01), a Matthews correlation coefficient (MCC) (0.89 ± 0.01), Cohen’s ĸ (0.89 ± 0.01), and mIoU (0.69 ± 0.01), with respective improvements of 4.05% in Macro F1, 1.89% in Micro F1, 0.09% in Weighted F1, 1.67% in MCC, 1.64% in Cohen’s ĸ, and 5.92% in Mean IoU over the second best model, SwinUNet; (2) Compared to the best two-source combinations (R + S, R + L), MTSCFNet achieved up to 1.50%, 3.28%, 3.42%, 6.72%, 6.76%, and 3.51% higher Macro F1, Micro F1, Weighted F1, MCC, Cohen’s ĸ, and mIoU, and up to 8.11%, 2.63%, 2.88%, 5.01%, 4.99%, and 11.48% improvements over single-source inputs, while also exhibiting the lowest variability, indicating strong robustness; (3) Under different fusion strategies, MTSCFNet with middle fusion surpassed early and late fusion by up to 15.31%, 3.74%, 3.99%, 7.66%, 7.76%, 22.33% and 24.13%, 5.76%, 6.20%, 11.48%, 11.57%, 32.96% in Macro F1, Micro F1, Weighted F1, MCC, Cohen’s ĸ, and mIoU, respectively, validating the effectiveness of feature-level multi-modal integration; (4) In cross-region transfer experiments, MTSCFNet demonstrated strong spatial generalizability, achieving average scores of 0.78 (Macro F1),0.87 (Micro F1), 0.86 (Weighted F1), 0.59 (MCC), 0.59 (Cohen’s ĸ), and 0.68 (mIoU), and outperformed SwinUNet by up to 38.80%, 9.40%, 18.58%, 22.48%, 26.17%, and 33.00% in Macro F1, Micro F1, Weighted F1, MCC, Cohen’s ĸ, and mIoU across varying forest densities. Overall, MTSCFNet offers a robust, accurate, and transferable solution for tree species classification in complex subtropical forest environments.

Graphical abstract

Keywords

Tree species classification / Deep learning / Multi-source remote sensing data / Subtropical forest

References

[1] Abbas S, Peng Q, Wong MS, Li Z, Wang J, Ng KTK, Kwok CYT, Hui KKW. Characterizing and classifying urban tree species using bi-monthly terrestrial hyperspectral images in Hong Kong. ISPRS J Photogramm Remote Sens, 2021, 177: 204-216.

[2] Alonzo M, Bookhagen B, Roberts DA. Urban tree species mapping using hyperspectral and lidar data fusion. Remote Sens Environ, 2014, 148: 70-83.

[3] Beloiu M, Heinzmann L, Rehush N, Gessler A, Griess VC. Individual tree-crown detection and species identification in heterogeneous forests using aerial RGB imagery and deep learning. Remote Sens, 2023, 15(5): 1463.

[4] Blickensdörfer L, Oehmichen K, Pflugmacher D, Kleinschmit B, Hostert P. National tree species mapping using Sentinel-1/2 time series and German National Forest Inventory data. Remote Sens Environ, 2024, 304114069

[5] Boisvenue C, White JC. Information needs of next-generation forest carbon models: opportunities for remote sensing science. Remote Sens, 2019, 11(4): 463.

[6] Cai Z, Wei H, Hu Q, Zhou W, Zhang X, Jin W, Wang L, Yu S, Wang Z, Xu B. Learning spectral-spatial representations from VHR images for fine-scale crop type mapping: a case study of rice-crayfish field extraction in South China. ISPRS J Photogramm Remote Sens, 2023, 199: 28-39.

[7] Cai Z, Xu B, Yu Q, Zhang X, Yang J, Wei H, Li S, Song Q, Xiong H, Wu H. A cost-effective and robust mapping method for diverse crop types using weakly supervised semantic segmentation with sparse point samples. ISPRS J Photogramm Remote Sens, 2024, 218: 260-276.

[8] Cao K, Zhang X. An improved res-unet model for tree species classification using airborne high-resolution images. Remote Sens, 2020, 12(7): 1128.

[9] Chang Z, Li H, Chen D, Liu Y, Zou C, Chen J, Han W, Liu S, Zhang N. Crop type identification using high-resolution remote sensing images based on an improved DeepLabV3+ network. Remote Sens, 2023, 15(21): 5088.

[10] Chen X, Shen X, Cao L. Tree species classification in subtropical natural forests using high-resolution UAV RGB and superview-1 multispectral imageries based on deep learning network approaches: a case study within the Baima snow mountain national nature reserve, China. Remote Sens, 2023, 15(102697

[11] Deur M, Gašparović M, Balenović I. An evaluation of pixel-and object-based tree species classification in mixed deciduous forests using pansharpened very high spatial resolution satellite imagery. Remote Sens, 2021, 1310): 1868.

[12] Fang F, McNeil BE, Warner TA, Maxwell AE. Combining high spatial resolution multi-temporal satellite data with leaf-on LiDAR to enhance tree species discrimination at the crown level. Int J Remote Sens, 2018, 39(23): 9054-9072.

[13] Fang F, McNeil BE, Warner TA, Maxwell AE, Dahle GA, Eutsler E, Li J. Discriminating tree species at different taxonomic levels using multi-temporal WorldView-3 imagery in Washington DC, USA. Remote Sens Environ, 2020, 246111811

[14] Fassnacht FE, Latifi H, Stereńczak K, Modzelewska A, Lefsky M, Waser LT, Straub C, Ghosh A. Review of studies on tree species classification from remotely sensed data. Remote Sens Environ, 2016, 186: 64-87.

[15] Feng B, Zheng C, Zhang W, Wang L, Yue C. Analyzing the role of spatial features when cooperating hyperspectral and LiDAR data for the tree species classification in a subtropical plantation forest area. J Appl Remote Sens, 2020, 14(2022213−022213

[16] Ferreira MP, De Almeida DRA, de Almeida Papa D, Minervino JBS, Veras HFP, Formighieri A, Santos CAN, Ferreira MAD, Figueiredo EO, Ferreira EJL. Individual tree detection and species classification of Amazonian palms using UAV images and deep learning. For Ecol Manage, 2020, 475118397

[17] Ferreira MP, Wagner FH, Aragão LE, Shimabukuro YE, de Souza Filho CR. Tree species classification in tropical forests using visible to shortwave infrared WorldView-3 images and texture analysis. ISPRS J Photogramm Remote Sens, 2019, 149: 119-131.

[18] Fricker GA, Ventura JD, Wolf JA, North MP, Davis FW, Franklin J. A convolutional neural network classifier identifies tree species in mixed-conifer forest from hyperspectral imagery. Remote Sens, 2019, 11(19): 2326.

[19] Grabska E, Frantz D, Ostapowicz K. Evaluation of machine learning algorithms for forest stand species mapping using Sentinel-2 imagery and environmental data in the Polish Carpathians. Remote Sens Environ, 2020, 251112103

[20] Guo Q, Zhang J, Guo S, Ye Z, Deng H, Hou X, Zhang H. Urban tree classification based on object-oriented approach and random forest algorithm using unmanned aerial vehicle (uav) multispectral imagery. Remote Sens, 2022, 1416): 3885.

[21] He T, Chen J, Pan D. GOFENet: a hybrid transformer-CNN network integrating GEOBIA-based object priors for semantic segmentation of remote sensing images. Remote Sens, 2025, 17(15): 2652.

[22] He P, Shen T, Wang Y, Zhu D, Hu Q, Li H, Yin W, Zhang Y, Yang A. A study on automatic annotation methods for watershed environmental elements based on semantic segmentation models. Eur J Remote Sens, 2025, 58(1): 2473939.

[23] Hermosilla T, Bastyr A, Coops NC, White JC, Wulder MA. Mapping the presence and distribution of tree species in Canada's forested ecosystems. Remote Sens Environ, 2022, 282113276

[24] Kattenborn T, Leitloff J, Schiefer F, Hinz S. Review on convolutional neural networks (CNN) in vegetation remote sensing. ISPRS J Photogramm Remote Sens, 2021, 173: 24-49.

[25] Li Y, Deng T, Fu B, Lao Z, Yang W, He H, Fan D, He W, Yao Y. Evaluation of decision fusions for classifying karst wetland vegetation using one-class and multi-class CNN models with high-resolution UAV images. Remote Sens, 2022, 14(22): 5869

[26] Liu W, Lin Y, Liu W, Yu Y, Li J. An attention-based multiscale transformer network for remote sensing image change detection. ISPRS J Photogramm Remote Sens, 2023, 202: 599-609.

[27] Liu Y, Chen D, Fu S, Mathiopoulos PT, Sui M, Na J, Peethambaran J. Segmentation of individual tree points by combining marker-controlled watershed segmentation and spectral clustering optimization. Remote Sens, 2024, 16(4): 610.

[28] Lu X, Jiang Z, Zhang H. Weakly supervised remote sensing image semantic segmentation with pseudo label noise suppression. IEEE Trans Geosci Remote Sens, 2024

[29] Mäyrä J, Keski-Saari S, Kivinen S, Tanhuanpää T, Hurskainen P, Kullberg P, Poikolainen L, Viinikka A, Tuominen S, Kumpula T. Tree species classification from airborne hyperspectral and LiDAR data using 3D convolutional neural networks. Remote Sens Environ, 2021, 256112322

[30] Modzelewska A, Fassnacht FE, Stereńczak K. Tree species identification within an extensive forest area with diverse management regimes using airborne hyperspectral data. Int J Appl Earth Obs Geoinf, 2020, 84101960

[31] Nezami S, Khoramshahi E, Nevalainen O, Pölönen I, Honkavaara E. Tree species classification of drone hyperspectral and RGB imagery with deep learning convolutional neural networks. Remote Sens, 2020, 12(7): 1070.

[32] Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B (2018) Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999. https://doi.org/10.48550/arXiv.1804.03999

[33] Osco LP, De Arruda MdS, Junior JM, Da Silva NB, Ramos APM, Moryia ÉAS, Imai NN, Pereira DR, Creste JE, Matsubara ET. A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS J Photogramm Remote Sens, 2020, 160: 97-106.

[34] Qin H, Zhou W, Yao Y, Wang W. Individual tree segmentation and tree species classification in subtropical broadleaf forests using UAV-based LiDAR, hyperspectral, and ultrahigh-resolution RGB data. Remote Sens Environ, 2022, 280113143

[35] Ronneberger O, Fischer P and Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Medical image computing and computer-assisted intervention–MICCAI 2015. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

[36] Roussel JR, Auty D, Coops NC, Tompalski P, Goodbody TR, Meador AS, Bourdon J-F, De Boissieu F, Achim A. lidR: an R package for analysis of Airborne Laser Scanning (ALS) data. Remote Sens Environ, 2020, 251112061

[37] Roussel JR, Auty D, De Boissieu F, Meadoret AS (2018) lidR: Airborne LiDAR data manipulation and visualization for forestry applications. R package version. 1(1)

[38] Shi C, Zhao X, Wang L. A multi-branch feature fusion strategy based on an attention mechanism for remote sensing image scene classification. Remote Sens, 2021, 13(10): 1950.

[39] Sicard P, Coulibaly F, Lameiro M, Araminiene V, De Marco A, Sorrentino B, Anav A, Manzini J, Hoshika Y, Moura BB. Object-based classification of urban plant species from very high-resolution satellite imagery. Urban for Urban Green, 2023, 81127866

[40] Terryn L, Calders K, Disney M, Origo N, Malhi Y, Newnham G, Raumonen P, Verbeeck H. Tree species classification using structural features derived from terrestrial laser scanning. ISPRS J Photogramm Remote Sens, 2020, 168: 170-181.

[41] Waldner F, Diakogiannis FI. Deep learning on edge: extracting field boundaries from satellite images with a convolutional neural network. Remote Sens Environ, 2020, 245111741

[42] Wang L, Zhang H, Lei K, Yang T, Zhang J, Cui Z, Fu R, Yu H, Zhao B, Wang X. A novel forest dynamic growth visualization method by incorporating spatial structural parameters based on convolutional neural network. IEEE J Sel Top Appl Earth Obs Remote Sens, 2023, 17: 3471-3488.

[43] Xi Z, Hopkinson C, Rood SB, Peddle DR. See the forest and the trees: effective machine and deep learning algorithms for wood filtering and tree species classification from terrestrial laser scanning. ISPRS J Photogramm Remote Sens, 2020, 168: 1-16.

[44] Xu Z, Shen X, Cao L, Coops NC, Goodbody TR, Zhong T, Zhao W, Sun Q, Ba S, Zhang Z. Tree species classification using UAS-based digital aerial photogrammetry point clouds and multispectral imageries in subtropical natural forests. Int J Appl Earth Obs Geoinf, 2020, 92102173

[45] Yu W, Gao L, Huang H, Shen Y, Shen G. HI 2 D 2 fnet: hyperspectral intrinsic image decomposition guided data fusion network for hyperspectral and LiDAR classification. IEEE Trans Geosci Remote Sens, 2023, 61: 1-15.

[46] Zhang C, Atkinson PM, George C, Wen Z, Diazgranados M, Gerard F. Identifying and mapping individual plants in a highly diverse high-elevation ecosystem using UAV imagery and deep learning. ISPRS J Photogramm Remote Sens, 2020, 169: 280-291.

[47] Zhang X, Su H, Zhang C, Gu X, Tan X, Atkinson PM. Robust unsupervised small area change detection from SAR imagery using deep learning. ISPRS J Photogramm Remote Sens, 2021, 173: 79-94.

RIGHTS & PERMISSIONS

The Author(s)