Potential of plant identification apps in urban forestry studies in China: comparison of recognition accuracy and user experience of five apps

Abstract

Information on species composition of an urban forest is essential for its management. However, to obtain this information becomes increasingly difficult due to limited taxonomic expertise. In this study, we tested the possibility of using plant identification applications running on mobile platforms to fill this vacuum. Five plant identification apps were compared for their potential in identifying urban tree species in China. An online survey was conducted to determine the features of apps that contributed to users’ satisfaction. The results show that identification accuracy varied significantly among the apps. The best performer achieved an accuracy of 74.6% at the species level, which is comparable to the accuracy by professionals in field surveys. Among the features of apps, accuracy of identification was the most important factor that contributed to users’ satisfaction. However, plant identification apps did not perform well when used on rare species or outside of the regions where they have been developed. Results indicate that plant identification apps have great potential in urban forest studies and management, but users need to be cautious when deciding which one to use.

Keywords

Plant identification / Mobile apps / Recognition accuracy / User satisfaction / Taxonomy

References

[1] Angelova A, Zhu S, Lin Y, Wong J, Shpecht C (2012) Development and deployment of a large-scale flower recognition mobile app. https://pdfs.semanticscholar.org/5ffd/d449a67615cc5457c1fbad5f5eb71d92765d.pdf. Accessed 03 Feb 2020

[2] Caballero C, Aranda MC (2012) WAPSI: web application for plant species identification using fuzzy image retrieval. In: Greco S, Bouchon-Meunier B, Coletti G, Fedrizzi M, Matarazzo B, Yager RR (eds) Proceedings of the14th international conference on information processing and management of uncertainty in knowledge-based systems. Springer, New York, pp 250–259

[3] Canfora G, Sorbo A, Mercaldo F, Visaggio CA (2016) Exploring mobile user experience through code quality metrics. In: Abrahamsson P, Jedlitschka A, Nguyen DA, Felderer M, Amasaki S, Mikkonen T (eds) Proceedings of the 17th international conference on product-focused software process improvement. Springer, New York, pp 705–712

[4] Carrus G, Scopelliti M, Lafortezza R, Colangelo G, Ferrini F, Salbitano F, Agrimi M, Portoghesi L, Sernenzato P, Sanesi G. Go greener, feel better? the positive effects of biodiversity on the well-being of individuals visiting, urban and peri-urban green areas. Landsc Urban Plan, 2015, 134: 221-228.

[5] Chang CC. Exploring mobile application customer loyalty: the moderating effect of use contexts. Telecommun Policy, 2015, 39(8): 678-690.

[6] Chen M, Liu X (2011) Predicting popularity of online distributed applications: iTunes app store case analysis. In: Proceedings of the 2011 iConference. ACM, New York, pp 661–663

[7] Cope JS, Corney D, Clark JY, Remagnino P, Wilkin P. Plant species identification using digital morphometrics: a review. Expert Syst Appl, 2012, 39(8): 7562-7573.

[8] Crall AW, Newman GJ, Stohlgren TJ, Holfelder KA, Graham J, Waller DM. Assessing citizen science data quality: an invasive species case study. Conserv Lett, 2011, 4(6): 433-442.

[9] Crown CA, Greer BZ, Gift DM, Watt FS. Every tree counts: reflections on NYC's third volunteer street tree inventory. Arboric Urban For, 2018, 44(2): 49-58.

[10] Çuğu İ, Şener E, Erciyes Ç, Balcı B, Akın E, Önal I, Akyüz AO (2017) Treelogy: a novel tree classifier utilizing deep and hand-crafted representations. https://arxiv.org/pdf/1701.08291.pdf. Accessed 04 Feb 2020

[11] Davis FD. User acceptance of information technology—system characteristics, user perceptions and behavioral impacts. Int J Man Mach Stud, 1993, 38(3): 475-487.

[12] Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: Xing EP, Jebara T (eds) Proceedings of the 31st international conference on machine learning. Microtome Publishing, Brookline, pp 647–655

[13] Du JX, Huang DS, Wang XF, Gu X. Computer-aided plant species identification (CAPSI) based on leaf shape matching technique. Trans Inst Meas Control, 2006, 28(3): 275-285.

[14] Du JX, Wang XF, Zhang GJ. Leaf shape based plant species recognition. Appl Math Comput, 2007, 185(2): 883-893.

[15] Escobedo FJ, Nowak DJ, Wagner JE, De la Maza CL, Rodríguez M, Crane DE, Hernández J. The socioeconomics and management of Santiago de Chile's public urban forests. Urban For Urban Green, 2006, 4(3–4): 105-114.

[16] Figueroa-Mata G, Mata-Montero E. Using a convolutional Siamese network for image-based plant species identification with small datasets. Biomimetics, 2020 5 1 8

[17] Foster J, Lowe A, Winkelman S. The value of green infrastructure for urban climate adaptation. Center Clean Air Policy, 2011, 750(1): 1-52.

[18] Fuller RA, Gaston KJ. The scaling of green space coverage in European cities. Biol Lett, 2009, 5(3): 352-355.

[19] Gao M, Lin Y, Sinnott RO (2017) A mobile application for plant recognition through deep learning. In: Proceedings of the 13th international conference on e-Science (e-Science). IEEE, New York, pp 29–38

[20] Goëau H, Bonnet P, Barbe J, Bakic V, Joly A, Molino JF, Barthelemy D, Boujemaa N (2012) Multi-organ plant identification. In: Proceedings of the 1st ACM international workshop on multimedia analysis for ecological data. ACM, New York, pp 41–44

[21] Goëau H, Bonnet P, Joly A, Bakić V, Barbe J, Yahiaoui I, Selmi S, Carré J, Barthélémy D, Boujemaa N, Molino JF (2013) Pl@ntNet mobile app. In: Proceedings of the 21st ACM international conference on multimedia. ACM, New York, pp 423–424

[22] Gray AN, Azuma DL. Repeatability and implementation of a forest vegetation indicator. Ecol Indic, 2005, 5(1): 57-71.

[23] Guerra García JM, Espinosa Torre F, García Gómez JC. Trends in taxonomy today: an overview about the main topics in taxonomy. Zool Baetica, 2008, 19: 15-49.

[24] Hawthorne TL, Elmore V, Strong A, Bennett-Martin P, Finnie J, Parkman J, Harris T, Singh J, Edwards L, Reed J. Mapping non-native invasive species and accessibility in an urban forest: a case study of participatory mapping and citizen science in Atlanta, Georgia. Appl Geogr, 2015, 56: 187-198.

[25] Hirokawa KH. Sustainability and the urban forest: an ecosystem services perspective. Nat Resour J, 2011, 51: 233-259.

[26] Joly A, Goëau H, Glotin H, Spampinato C, Bonnet P, Vellinga W, Planque R, Rauber A, Fisher R, Müller H (2014) LifeCLEF 2014: multimedia life species identification challenges. In: Kanoulas E (eds) Proceedings of the 5th international conference of the cross-language evaluation forum for European languages. Springer, New York, pp 229–249

[27] Kapoor AP, Vij M. How to boost your app store rating? An empirical assessment of ratings for mobile banking apps. J Theor Appl Electron Commer Res, 2020, 15(1): 99-115.

[28] Keivani M, Mazloum J, Sedaghatfar E, Tavakoli MB. Automated analysis of leaf shape, texture, and color features for plant classification. Trait Signal, 2020, 37(1): 17-28.

[29] Kendal D, Dobbs C, Lohr VI. Global patterns of diversity in the urban forest: Is there evidence to support the 10/20/30 rule?. Urban For Urban Green, 2014, 13(3): 411-417.

[30] Kim JH, Huang RG, Jin SH, Hong KS (2009) Mobile-based flower recognition system. In: Proceedings of the 2009 third international symposium on intelligent information technology application. IEEE, New York, pp 580–583

[31] Korotaeva D, Khlopotov M, Makarenko A, Chikshova E, Startseva N, Chemysheva A (2018) Botanicum: a telegram bot for tree classification. In: Proceedings of the 2018 22nd conference of open innovations association (FRUCT). IEEE, New York, pp 88–93

[32] Kumar N, Belhumeur PN, Biswas A, Jacobs DW, Kress WJ, Lopez IC, Soares JV (2012) Leafsnap: a computer vision system for automatic plant species identification. In: Proceedings of the European conference on computer vision. Springer, New York, pp 502–516

[33] Lee S, Lee D. What is the proper way to apply the multiple comparison test?. Korean J Anesthesiol, 2018, 71(5): 353-360.

[34] Lefland AB, Huff ES, Donahue B. A community forestry model linking research, management, education, and stakeholder engagement: case study results from the town of Weston, Massachusetts, USA. Small Scale For, 2018, 17(2): 191-210.

[35] McPherson EG, Nowak D, Heisler G, Grimmond S, Souch C, Grant R, Rowntree R. Quantifying urban forest structure, function, and value: the Chicago urban forest climate project. Urban Ecosyst, 1997, 1(1): 49-61.

[36] Meulman JJ, Heiser WJ. IBM SPSS Categories 19, 1989, Chicago: SPSS Inc. p109

[37] Newmaster SG, Ragupathy S, Janovec J. A botanical renaissance: state-of-the-art DNA bar coding facilitates an automated identification technology system for plants. Int J Comput Appl Technol, 2009, 35(1): 50-60.

[38] Nowak DJ, Crane DE, Stevens JC, Hoehn RE, Walton JT, Bond J. A ground-based method of assessing urban forest structure and ecosystem services. Arboric Urban For, 2008, 34(6): 347-358.

[39] Nowak DJ, Walton JT. Projected urban growth (2000–2050) and its estimated impact on the US forest resource. J For, 2005, 103(8): 383-389.

[40] Romana LA, Scharenbrochb BC, Ostberg JPA, Muellere LS, Henningg JG, Koeserh AK, Sandersi JR, Betza DR, Jordanj RC. Data quality in citizen science urban tree inventories. Urban For Urban Green, 2017, 22: 124-135.

[41] Saitoh T, Kaneko T (2000) Automatic recognition of wild flowers. In: Proceedings of the 15th international conference on pattern recognition. IEEE, New York, pp 507–510

[42] Seiwa K. Ontogenetic changes in leaf phenology of Ulmus davidiana var. japonica, a deciduous broad-leaved tree. Tree Physiol, 1999, 19(12): 793-797.

[43] Sharif Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. IEEE, New York, pp 806–813

[44] Shrode F. Mobile apps for nature field guides. Ref Rev, 2012, 26(7): 4-6.

[45] Simmons A, Hoon L (2016) Agree to disagree: on labelling helpful app reviews. In: Proceedings of the 28th Australian computer–human interaction conference on computer–human interaction. ACM, New York, pp 416–420

[46] Söderkvist O (2001) Computer vision classification of leaves from Swedish trees. https://www.diva-portal.org/smash/get/diva2:303038/FULLTEXT01.pdf. Accessed 04 Feb 2019

[47] Son J. The impact of initial experience and user attachment on application downloads information-seeking and—sharing applications. Internet Res, 2017, 27(2): 256-276.

[48] Song J (2019) According to China's “flower identification” big data analysis report forty million users have identified plants with AI technology over 500 million times in total. https://finance.chinadevelopment.com.cn/hy/wy/2019/04/1501723.shtml. Accessed 03 Aug 2019

[49] Tan JW, Chang SW, Abdul-Kareem S, Yap HJ, Yong KT. Deep learning for plant species classification using leaf vein morphometric. IEEE ACM Trans Comput Biol Bioinform, 2020, 17(1): 82-90.

[50] United Nations. World urbanization prospects: the 2018 revision, 2018, New York: UN 56

[51] Valentini A, Pompanon F, Taberlet P. DNA barcoding for ecologists. Trends Ecol Evol, 2009, 24(2): 110-117.

[52] van Der Kooij AJ, Meulman JJ, Heiser WJ. Local minima in categorical multiple regression. Comput Stat Data Anal, 2006, 50(2): 446-462.

[53] Vasa R, Hoon L, Mouzakis K, Noguchi A (2012) A preliminary analysis of mobile app user reviews. In: Proceedings of the 24th Australian computer–human interaction conference. ACM, New York, pp 241–244

[54] Wägele H, Klussmann-Kolb A, Kuhlmann M, Haszprunar G, Lindberg D, Koch A, Wägele JW. The taxonomist-an endangered race. A practical proposal for its survival. Front Zool, 2011 8 1 25

[55] Wang B, Brown D, Gao Y, La Salle J (2013) Mobile plant leaf identification using smart-phones. In: Proceedings of the 2013 ieee international conference on image processing. IEEE, New York, pp 4417–4421

[56] Wang GM, Zuo JC, Li XR, Liu YH, Yu JB, Shao HB, Li YZ. Low plant diversity and floristic homogenization in fast-urbanizing towns in Shandong peninsular, China: effects of urban greening at regional scale for ecological engineering. Ecol Eng, 2014, 64: 179-185.

[57] Wang XF, Huang DS, Du JX, Xu H, Heutte L. Classification of plant leaf images with complicated background. Appl Math Comput, 2008, 205(2): 916-926.

[58] Winberg S, Katz S, Mishra AK (2013) Fynbos leaf online plant recognition application. In: Proceedings of the 2013 fourth national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG). IEEE, New York, pp 66–84

[59] Wood E, Harsant A, Dallimer M, de Chavez AC, McEachan RRC, Hassall C. Not all green space is created equal: biodiversity predicts psychological restorative benefits from urban green space. Front Psychol, 2018, 9: 13.

[60] Ye Y, Chen C, Li CT, Fu H, Chi Z (2004) A computerized plant species recognition system. In: Proceedings of 2004 international symposium on intelligent multimedia, video and speech processing. IEEE, New York, pp 723–726

[61] Yoon VY, Hostler RE, Guo Z, Guimaraes T. Assessing the moderating effect of consumer product knowledge and online shopping experience on using recommendation agents for customer loyalty. Decis Support Syst, 2013, 55(4): 883-893.