Machine learning-based crop recognition from aerial remote sensing imagery

Abstract

Timely and accurate acquisition of crop distribution and planting area information is important for making agricultural planning and management decisions. This study employed aerial imagery as a data source and machine learning as a classification tool to statically and dynamically identify crops over an agricultural cropping area. Comparative analysis of pixel-based and object-based classifications was performed and classification results were further refined based on three types of object features (layer spectral, geometry, and texture). Static recognition using layer spectral features had the highest accuracy of 75.4% in object-based classification, and dynamic recognition had the highest accuracy of 88.0% in object-based classification based on layer spectral and geometry features. Dynamic identification could not only attenuate the effects of variations on planting dates and plant growth conditions on the results, but also amplify the differences between different features. Object-based classification produced better results than pixel-based classification, and the three feature sets (layer spectral alone, layer spectral and geometry, and all three) resulted in only small differences in accuracy in object-based classification. Dynamic recognition combined with object-based classification using layer spectral and geometry features could effectively improve crop classification accuracy with high resolution aerial imagery. The methodologies and results from this study should provide practical guidance for crop identification and other agricultural mapping applications.

Keywords

machine learning / crop recognition / aerial imagery / dynamic recognition / static recognition

References

[1] Bauer M E, Cipra J E (1973). Identification of agricultural crops by computer processing of ERTS MSS data. In: The Proceedings of the Symposium on Significant Results Obtained from the Earth Resources Technology Satellite-1. New Carollton, IN, USA, 3, 205–212

[2] Benz U C, Hofmann P, Willhauck G, Lingenfelder I, Heynen M (2004). Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information. ISPRS J Photogramm Remote Sens, 58(3–4): 239–258

[3] Boryan C, Yang Z, Mueller R, Craig M (2011). Monitoring US agriculture: the US Department of Agriculture, National Agricultural Statistics Service, Cropland Data Layer Program. Geocarto Int, 26(5): 341–358

[4] Breiman L, Friedman J, Stone C J, Olshen R (1984). Classification and Regression Trees. New York: Wadsworth Inc

[5] Cai Y, Guan K, Peng J, Wang S, Seifert C, Wardlow B, Li Z (2018). A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote Sens Environ, 210: 35–47

[6] Camargo Neto J (2004). A combined statistical-soft computing approach for classification and mapping weed species in minimum-tillage systems. Dissertation for the Doctoral Degree. Lincoln: University of Nebraska

[7] Chubey M S, Franklin S E, Wulder M A (2006). Object-based analysis of Ikonos-2 imagery for extraction of forest inventory parameters. Photogramm Eng Remote Sensing, 72(4): 383–394

[8] Congalton R G (1991). A review of assessing the accuracy of classifications of remotely sensed data. Remote Sens Environ, 37(1): 35–46

[9] Damian J M, Pias O H de C, Cherubin M R, da Fonseca A Z, Fornari E Z, Santi A L (2020). Applying the NDVI from satellite images in delimiting management zones for annual crops. Sci Agric, 77(1): e20180055

[10] Dimov D, Löw F, Uhl J H, Kenjabaev S, Dubovyk O, Ibrakhimov M, Biradar C (2019). Framework for agricultural performance assessment based on MODIS multitemporal data. J Appl Remote Sens, 13(2): 1

[11] Drăguţ L, Csillik O, Eisank C, Tiede D (2014). Automated parameterisation for multi-scale image segmentation on multiple layers. ISPRS J Photogramm Remote Sens, 88(100): 119–127

[12] eCognition (2019). User Guide. Available at eCognition website

[13] Farg E, Ramadan M N, Arafat S M (2019). Classification of some strategic crops in Egypt using multi-remotely sensing sensors and time series analysis. Egypt J Remote Sens Space Sci, 22(3): 263–270

[14] Foody G M (2009). Classification accuracy comparison: Hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority. Remote Sens Environ, 113(8): 1658–1663

[15] Hossain M D, Chen D (2019). Segmentation for Object-Based Image Analysis (OBIA): a review of algorithms and challenges from remote sensing perspective. ISPRS J Photogramm Remote Sens, 150: 115–134

[16] Hu Q, Wu W B, Song Q, Lu M, Chen D, Yu Q Y, Tang H J (2017). How do temporal and spectral features matter in crop classification in Heilongjiang Province, China? J Integr Agric, 16(02): 324–336

[17] Huete A R (1988). A soil-adjusted vegetation index (SAVI). Remote Sens Environ, 25(3): 295–309

[18] Hughes G F (1968). On the mean accuracy of statistical pattern recognizers. IEEE Trans Inf Theory, 14(1): 55–63

[19] Jakubauskas M E, Legates D R, Kastens J H (2002). Crop identification using harmonic analysis of time-series AVHRR NDVI data. Comput Electron Agric, 37(1–3): 127–139

[20] Jensen J R (2005). Introductory Digital Image Processing: A Remote Sensing Perspective. New Jersey: Prentice-Hall, Inc

[21] Jordan C F (1969). Derivation of leaf-area index from quality of light on the forest floor. Ecological Society of America, 50(4): 663–666

[22] Kenduiywo B K, Bargiel D, Soergel U (2016). Crop type mapping from a sequence of terrasar-X images with dynamic conditional random fields. ISPRS Annals of the Photogrammetry. Remote Sensing and Spatial Information Sciences, 3(7): 59–66

[23] Knight J F, Lunetta R S, Ediriwickrema J, Khorram S (2006). Regional scale land cover characterization using MODIS-NDVI 250 m multi-temporal imagery: a phenology-based approach. GIsci Remote Sens, 43(1): 1–23

[24] Laliberte A S, Browning D M, Rango A (2012). A comparison of three feature selection methods for object-based classification of sub-decimeter resolution UltraCam-L imagery. Int J Appl Earth Obs Geoinf, 15: 70–78

[25] Lambert M J, Traoré P C S, Blaes X, Baret P, Defourny P (2018). Estimating smallholder crops production at village level from Sentinel-2 time series in Mali’s cotton belt. Remote Sens Environ, 216: 647–657

[26] Lichtblau E, Oswald C J (2019). Classification of impervious land-use features using object-based image analysis and data fusion. Comput Environ Urban Syst, 75: 103–116

[27] Löw F, Michel U, Dech S, Conrad C (2013). Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using Support Vector Machines. ISPRS J Photogramm Remote Sens, 85: 102–119

[28] Ma L, Li M, Ma X, Cheng L, Du P, Liu Y (2017). A review of supervised object-based land-cover image classification. ISPRS J Photogramm Remote Sens, 130: 277–293

[29] Masialeti I, Egbert S, Wardlow B D (2010). A comparative analysis of phenological curves for major crops in Kansas. GIsci Remote Sens, 47(2): 241–259

[30] Meyer G E, Hindman T, Laksmi K (1999). Machine vision detection parameters for plant species identification. Proc SPIE, 3543: 327–335

[31] Murmu S, Biswas S (2015). Application of fuzzy logic and neural network in crop classification: a review. Aquatic Procedia, 4: 1203–1210

[32] Myint S W, Gober P, Brazel A, Grossman-Clarke S, Weng Q H (2011). Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery. Remote Sens Environ, 115(5): 1145–1161

[33] Odenweller J B, Johnson K I (1984). Crop identification using Landsat temporal-spectral profiles. Remote Sens Environ, 14(1–3): 39–54

[34] Mutanga O, Dube T, Galal O(2017). Remote sensing of crop health for food security in Africa: potentials and constraints. Remote Sensing Applications: Society and Environment, 8: 231–239

[35] Pal M (2013). Hybrid genetic algorithm for feature selection with hyperspectral data. Remote Sens Lett, 4(7): 619–628

[36] Pal M, Foody G M (2010). Feature selection for classification of hyperspectral data by SVM. IEEE Trans Geosci Remote Sens, 48(5): 2297–2307

[37] Peña M A, Brenning A (2015). Assessing fruit-tree crop classification from Landsat-8 time series for the Maipo Valley, Chile. Remote Sens Environ, 171: 234–244

[38] Richards J A, Jia X (2006). Remote Sensing Digital Image Analysis, 3rd ed. Berlin: Springer-Verlag, 273–274.

[39] Richardson A J, Everitt J H (1992). Using spectral vegetation indices to estimate rangeland productivity. Geocarto Int, 7(1): 63–69

[40] Rouse J W Jr, Haas R H, Schell J A, Deering D W (1974). Monitoring vegetation systems in the great plains with ERTS. In: Proceedings of the Third ERTS-1 Symposium NASA, NASA SP-351. Washington: 309–317

[41] Rondeaux G, Steven M, Baret F (1996). Optimization of soil-adjusted vegetation indices. Remote Sens Environ, 55(2): 95–107

[42] Sakamoto T, Gitelson A A, Nguy-Robertson A L, Arkebauer T J, Wardlow B D, Suyker A E, Verma S B, Shibayama M (2012). An alternative method using digital cameras for continuous monitoring of crop status. Agric Meteorol, 154–155: 113–126

[43] Shen Y, Chen J, Xiao L, Pan D (2019). Optimizing multiscale segmentation with local spectral heterogeneity measure for high resolution remote sensing images. ISPRS J Photogramm Remote Sens, 157: 13–25

[44] Siachalou S, Mallinis G, Tsakiri-Strati M (2015). A hidden markov models approach for crop classification: linking crop phenology to time series of multi-sensor remote sensing data. Remote Sens, 7(4): 3633–3650

[45] Sibanda M, Murwira A (2012). The use of multi-temporal MODIS images with ground data to distinguish cotton from maize and sorghum fields in smallholder agricultural landscapes of Southern Africa. Int J Remote Sens, 33(16): 4841–4855

[46] Song H, Yang C, Zhang J, Hoffmann W C, He D, Thomasson J A (2016). Comparison of mosaicking techniques for airborne images from consumer-grade cameras. J Appl Remote Sens, 10(1): 016030

[47] Torres-Sánchez J, Peña J M, de Castro A I, López-Granados F (2014). Multi-temporal mapping of the vegetation fraction in early-season wheat fields using images from UAV. Comput Electron Agric, 103: 104–113

[48] van Klompenburg T, Kassahun A, Catal C (2020). Crop yield prediction using machine learning: A systematic literature review. Comput Electron Agric, 177: 105709

[49] van Niel T G, McVicar T R (2004). Determining temporal windows for crop discrimination with remote sensing: a case study in south-eastern Australia. Comput Electron Agric, 45(1–3): 91–108

[50] Waldhoff G, Lussem U, Bareth G (2017). Multi-Data Approach for remote sensing-based regional crop rotation mapping: a case study for the Rur catchment, Germany. Int J Appl Earth Obs Geoinf, 61: 55–69

[51] Wang P, Fan E, Wang P (2021). Comparative analysis of image classification algorithms based on traditional machine learning and deep learning. Pattern Recognit Lett, 141: 61–67

[52] Woebbecke D M, Meyer G E, Von Bargen K, Mortensen D A, Woebbecke D M, Meyer G E,von Bargen K , Mortensen D A (1995). Color indices for weed identification under various soil, residue, and lighting conditions. Trans ASAE, 38(1): 259–269

[53] Wu B, Meng J, Li Q, Yan N, Du X, Zhang M (2014). Remote sensing-based global crop monitoring: experiences with China’s CropWatch system. Int J Digit Earth, 7(2): 113–137

[54] Wu M, Yang C, Song X, Hoffmann W C, Huang W, Niu Z, Wang C, Li W, Yu B (2018). Monitoring cotton root rot by synthetic Sentinel-2 NDVI time series using improved spatial and temporal data fusion. Sci Rep, 8(1): 2016

[55] Yang C, Everitt J H, Murden D (2011). Evaluating high resolution SPOT 5 satellite imagery for crop identification. Comput Electron Agric, 75(2): 347–354

[56] Yang C, Hoffmann W C (2015). Low-cost single-camera imaging system for aerial applicators. J Appl Remote Sens, 9(1): 096064

[57] Yu Q, Gong P, Clinton N, Biging G, Kelly M, Schirokauer D (2006). Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery. Photogramm Eng Remote Sensing, 72(7): 799–811

[58] Zhang X, Xiao P, Song X, She J (2013). Boundary-constrained multi-scale segmentation method for remote sensing images. ISPRS J Photogramm Remote Sens, 78: 15–25

[59] Zhang J, Feng L, Yao F (2014). Improved maize cultivated area estimation over a large scale combining MODIS–EVI time series data and crop phenological information. ISPRS J Photogramm Remote Sens, 94: 102–113

[60] Zhang J, Yang C, Song H, Hoffmann W, Zhang D, Zhang G (2016). Evaluation of an airborne remote sensing platform consisting of two consumer-grade cameras for crop identification. Remote Sens, 8(3): 257

[61] Zheng B, Myint S W, Thenkabail P S, Aggarwal R M (2015). A support vector machine to identify irrigated crop types using time-series Landsat NDVI data. Int J Appl Earth Obs Geoinf, 34: 103–112

[62] Zhong L, Gong P, Biging G S (2014). Efficient corn and soybean mapping with temporal extendability: a multi-year experiment using Landsat imagery. Remote Sens Environ, 140: 1–13

[63] Zhou F, Zhang A, Townley-Smith L (2013). A data mining approach for evaluation of optimal time-series of MODIS data for land cover mapping at a regional level. ISPRS J Photogramm Remote Sens, 84: 114–129

Acknowledgments

This work was supported by the National Key Research and Development Program (No. 2020YFD1100204), and the Provincial Key Basic Research Project (No. 2019AB002). The authors wish to thank Lee Denham and Fred Gomez at USDA-ARS in College Station, Texas for the collection of the airborne images. Mention of a commercial product is solely for the purpose of providing specific information and should not be construed as a product endorsement by the authors or the institutions with which the authors are affiliated.

RIGHTS & PERMISSIONS

2021 Higher Education Press