Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

Abstract

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification. The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles (UAVs) provides a new research direction for urban tree species classification. We proposed an RGB optical image dataset with 10 urban tree species, termed TCC10, which is a benchmark for tree canopy classification (TCC). TCC10 dataset contains two types of data: tree canopy images with simple backgrounds and those with complex backgrounds. The objective was to examine the possibility of using deep learning methods (AlexNet, VGG-16, and ResNet-50) for individual tree species classification. The results of convolutional neural networks (CNNs) were compared with those of K-nearest neighbor (KNN) and BP neural network. Our results demonstrated: (1) ResNet-50 achieved an overall accuracy (OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16. (2) The classification accuracy of KNN and BP neural network was less than 70%, while the accuracy of CNNs was relatively higher. (3) The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds. For the deciduous tree species in TCC10, the classification accuracy of ResNet-50 was higher in summer than that in autumn. Therefore, the deep learning is effective for urban tree species classification using RGB optical images.

Keywords

Urban forest / Unmanned aerial vehicle (UAV) / Convolutional neural network / Tree species classification / RGB optical images

References

[1] Alonzo M, Bookhagen B, Roberts DA. Urban tree species mapping using hyperspectral and lidar data fusion. Remote Sens Environ, 2014, 148: 70-83.

[2] Atito S, Yanikoglu B, Aptoula E (2017) Plant identification with large number of classes: Sabanciu-gebzetu system in plantclef 2017. In: Conference and labs of the evaluation forum, Dublin, Ireland, 11–14 September

[3] Dalponte M, Bruzzone L, Gianelle D. Tree species classification in the southern alps based on the fusion of very high geometrical resolution multispectral/hyperspectral images and lidar data. Remote Sens Environ, 2012, 123: 258-270.

[4] Dalponte M, Orka HO, Gobakken T, Gianelle D, Næsset E. Tree species classification in boreal forests with hyperspectral data. IEEE Trans Geosci Remote Sens, 2013, 51: 2632-2645.

[5] Dalponte M, Ørka HO, Ene LT, Gobakken T, Næsset E. Tree crown delineation and tree species classification in boreal forests using hyperspectral and ALS data. Remote Sens Environ, 2014, 140: 306-317.

[6] Dalponte M, Ene LT, Marconcin M, Gobakken T, Næsset E. Semi-supervised SVM for individual tree crown species classification. ISPRS J Photogramm Remote Sens, 2015, 110: 77-87.

[7] Dian Y, Pang Y, Dong Y, Li Z. Urban tree species mapping using airborne LiDAR and hyperspectral data. J Indian Soc Remote Sens, 2016, 44: 595-603.

[8] Endreny TA. Strategically growing the urban forest will improve our world. Nat Commun, 2018, 9: 1160.

[9] Fassnacht FE, Latifi H, Stereńczak K, Modzelewska A, Lefsky M, Waser LT, Straub C, Ghosh A. Review of studies on tree species classification from remotely sensed data. Remote Sens Environ, 2016, 186: 64-87.

[10] Hartling S, Sagan V, Sidike P, Maimaitijiang M, Carron J. Urban tree species classification using a WorldView-2/3 and LiDAR data fusion approach and deep learning. Sensors, 2019, 19: 1284.

[11] He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, Las Vegas, USA, 26 June–1 July

[12] Huang H, Li X, Chen C. Individual tree crown detection and delineation from very-high-resolution UAV images based on bias field and marker-controlled watershed segmentation algorithms. IEEE J Sel Top Appl Earth Obs Remote Sens, 2018, 11: 1-10.

[13] Jones TG, Coops NC, Sharma T. Assessing the utility of airborne hyperspectral and LiDAR data for species distribution mapping in the coastal Pacific Northwest, Canada. Remote Sens Environ, 2010, 114: 2841-2852.

[14] Koch B, Heyder U, Weinacker H. Detection of individual tree crowns in airborne lidar data. Photogramm Eng Remote Sens, 2006, 72: 357-363.

[15] Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, Lake Tahoe, USA, 3–6 December

[16] Lin Y, Jiang M, Yao Y, Zhang L, Lin J. Use of UAV oblique imaging for the detection of individual trees in residential environments. Urban For Urban Greening, 2015, 14: 404-412.

[17] Liu H, Wu C. Crown-level tree species classification from AISA hyperspectral imagery using an innovative pixel-weighting approach. Int J Appl Earth Obs Geoinf, 2018, 68: 298-307.

[18] Liu L, Coops NC, Aven NW, Pang Y. Mapping urban tree species using integrated airborne hyperspectral and LiDAR remote sensing data. Remote Sens Environ, 2017, 200: 170-182.

[19] Liu T, Li R, Zhong X, Jiang M, Jin X, Zhou P, Liu S, Sun C, Guo W. Estimates of rice lodging using indices derived from UAV visible and thermal infrared images. Agric For Meteorol, 2018, 252: 144-154.

[20] Maschler J, Atzberger C, Immitzer M. Individual tree crown segmentation and classification of 13 tree species using airborne hyperspectral data. Remote Sens, 2018, 10: 1218.

[21] Moskal LM, Styers DM, Halabisky M. Monitoring urban tree cover using object-based image analysis and public domain remotely sensed data. Remote Sens, 2011, 3: 2243-2262.

[22] Mozgeris G, Juodkien V, Jonikavicius D, Straigyte L, Gadal S, Ouerghemmi W. Ultra-light aircraft-based hyperspectral and colour-infrared imaging to identify deciduous tree species in an urban environment. Remote Sens, 2018, 10: 1668.

[23] Nevalainen O, Honkavaara E, Tuominen S, Viljanen N, Hakala T, Yu X, Hyyppä J, Saari H, Pölönen I, Imai N, Tommaselli AM. Individual tree detection and classification with UAV-based photogrammetric point clouds and hyperspectral imaging. Remote Sens, 2017, 9: 185.

[24] Pu R, Landry S, Yu Q. Assessing the potential of multi-seasonal high resolution Pléiades satellite imagery for mapping urban tree species. Int J Appl Earth Obs Geoinf, 2018, 71: 144-158.

[25] Puliti S, Ørka H, Gobakken T, Næsset E. Inventory of small forest areas using an unmanned aerial system. Remote Sens, 2015, 7: 9632-9654.

[26] Ren Z, Pu R, Zheng H, Zhang D, He X. Spatiotemporal analyses of urban vegetation structural attributes using multitemporal Landsat TM data and field measurements. Ann For Sci, 2017, 74: 54.

[27] Ren Z, Du Y, He X, Pu R, Zheng H, Hu H. Spatiotemporal pattern of urban forest leaf area index in response to rapid urbanization and urban greening. J For Res, 2018, 29(3): 785-796.

[28] Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning repre-sentations, San Diego, USA, 7–9 May

[29] Swain KC, Thomson SJ, Jayasuriya HPW. Adoption of an unmanned helicopter for low-altitude remote sensing to estimate yield and total biomass of a rice crop. Trans ASABE, 2010, 53: 21-27.

[30] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, Boston, USA, 7–12 June

[31] Xie Y, Bao H, Shekhar S, Knight J (2018) A TIMBER framework for mining urban tree inventories using remote sensing datasets. In: IEEE international conference on data mining, Singapore, 17–20 November