Growth monitoring of greenhouse lettuce based on a convolutional neural network

Abstract

Growth-related traits, such as aboveground biomass and leaf area, are critical indicators to characterize the growth of greenhouse lettuce. Currently, nondestructive methods for estimating growth-related traits are subject to limitations in that the methods are susceptible to noise and heavily rely on manually designed features. In this study, a method for monitoring the growth of greenhouse lettuce was proposed by using digital images and a convolutional neural network (CNN). Taking lettuce images as the input, a CNN model was trained to learn the relationship between images and the corresponding growth-related traits, i.e., leaf fresh weight (LFW), leaf dry weight (LDW), and leaf area (LA). To compare the results of the CNN model, widely adopted methods were also used. The results showed that the values estimated by CNN had good agreement with the actual measurements, with R² values of 0.8938, 0.8910, and 0.9156 and normalized root mean square error (NRMSE) values of 26.00, 22.07, and 19.94%, outperforming the compared methods for all three growth-related traits. The obtained results showed that the CNN demonstrated superior estimation performance for the flat-type cultivars of Flandria and Tiberius compared with the curled-type cultivar of Locarno. Generalization tests were conducted by using images of Tiberius from another growing season. The results showed that the CNN was still capable of achieving accurate estimation of the growth-related traits, with R² values of 0.9277, 0.9126, and 0.9251 and NRMSE values of 22.96, 37.29, and 27.60%. The results indicated that a CNN with digital images is a robust tool for the monitoring of the growth of greenhouse lettuce.

References

[1] Teobaldelli, M. et al. Developing an accurate and fast non-destructive single leaf area model for loquat (Eriobotrya japonica Lindl) cultivars. Plants 8, 1-12 (2019).

[2] Lati, R. N., Filin, S. & Eizenberg, H. Estimation of plants’ growth parameters via image-based reconstruction of their three-dimensional shape. Agron. J. 105, 191-198 (2013).

[3] Bauer, A. et al. Combining computer vision and deep learning to enable ultra-scale aerial phenotyping and precision agriculture: a case study of lettuce production. Hortic. Res. 6, 1-12 (2019).

[4] Levy, P. E. & Jarvis, P. G. Direct and indirect measurements of LAI in millet and fallow vegetation in HAPEX-Sahel. Agric. Meteorol. 97, 199-212 (1999).

[5] Casadesús, J. & Villegas, D. Conventional digital cameras as a tool for assessing leaf area index and biomass for cereal breeding. Plant Biol. 56, 7-14 (2014).

[6] Chen, D. et al. Predicting plant biomass accumulation from image-derived parameters. Gigascience 7, 5-27 (2018).

[7] Zhang, L., Verma, B., Stockwell, D. & Chowdhury, S. Density weighted connectivity of grass pixels in image frames for biomass estimation. Expert Syst. Appl. 101, 213-227 (2018).

[8] Hu, Y., Wang, L., Xiang, L., Wu, Q. & Jiang, H. Automatic non-destructive growth measurement of leafy vegetables based on kinect. Sensors 18, 1-23 (2018).

[9] Krishna, G., Sahoo, R. N., Singh, P. & Bajpai, V. Comparison of various modelling approaches for water deficit stress monitoring in rice crop through hyperspectral remote sensing. Agric Water Manag. 213, 231-244 (2019).

[10] Yu, J., Li, C . & Paterson, A. H. High-throughput phenotyping of cotton plant height using depth images under field conditions. Comput. Electron. Agric 130, 57-68 (2016).

[11] Tackenberg, O. A new method for non-destructive measurement of biomass, growth rates, vertical biomass distribution and dry matter content based on digital image analysis. Ann. Bot. 99, 777-783 (2007).

[12] Fan, X. et al. A simple visible and near-infrared (V-NIR) camera system for monitoring the leaf area index and growth stage of Italian ryegrass. Comput. Electron. Agric. 144, 314-323 (2018).

[13] Liu, J. & Pattey, E. Retrieval of leaf area index from top-of-canopy digital photography over agricultural crops. Agric. Meteorol. 150, 1485-1490 (2010).

[14] Sakamoto, T. et al. Application of day and night digital photographs for estimating maize biophysical characteristics. Precis. Agric. 13, 285-301 (2012).

[15] Ma, J. et al. A segmentation method for greenhouse vegetable foliar disease spots images using color information and region growing. Comput. Electron. Agric. 142, 110-117 (2017).

[16] Wan, J., Wang, D., Hoi, S.C.H. & Wu, P. Deep learning for content-based image retrieval: a comprehensive study. In Proc. 22nd ACM International Conference on Multimedia, 157-166 (Istanbul, Turkey, 2014).

[17] Pound, M. P. et al. Deep machine learning provides state-of-the-art performance in image-based plant phenotyping. Gigascience 6, 1-10 (2017).

[18] Ma, J. et al. Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network. Eur. J. Agron. 103, 117-129 (2019).

[19] Ferreira, S., Freitas, D. M., Gonçalves, G., Pistori, H. & Theophilo, M. Weed detection in soybean crops using ConvNets. Comput. Electron. Agric 143, 314-324 (2017).

[20] Ghosal, S. et al. An explainable deep machine vision framework for plant stress phenotyping. Proc. Natl Acad. Sci. 115, 4613-4618 (2018).

[21] Uzal, L. C. et al. Seed-per-pod estimation for plant breeding using deep learning. Comput. Electron. Agric. 150, 196-204 (2018).

[22] Dyrmann, M., Karstoft, H . & Midtiby, H. S. Plant species classification using deep convolutional neural network. Biosyst. Eng. 151, 72-80 (2016).

[23] Grinblat, G. L., Uzal, L. C., Larese, M. G. & Granitto, P. M. Deep learning for plant identification using vein morphological patterns. Comput. Electron. Agric. 127, 418-424 (2016).

[24] Nachtigall, L. G., Araujo, R. M. & Nachtigall, G. R. Classification of apple tree disorders using convolutional neural networks. In Proc. 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (San Jose, California, 2016).

[25] Mohanty, S. P., Hughes, D. P. & Salathé, M. Using deep learning for image-based plant disease detection. Front Plant Sci. 7, 1-10 (2016).

[26] Ferentinos, K. P. Deep learning models for plant disease detection and diagnosis. Comput. Electron. Agric. 145, 311-318 (2018).

[27] Ramcharan, A. et al. Deep learning for image-based cassava disease detection. Front Plant Sci. 8, 1-7 (2017).

[28] Ma, J. et al. A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network. Comput. Electron. Agric. 154, 18-24 (2018).

[29] Ubbens, J., Cieslak, M., Prusinkiewicz, P. & Stavness, I. The use of plant models in deep learning: an application to leaf counting in rosette plants. Plant Methods 14, 1-10 (2018).

[30] Zhang, Z. Microsoft kinect sensor and its effect. IEEE Multimed. 19, 4-10 (2012).

[31] Xiong, X. et al. Panicle-SEG: a robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization. Plant Methods 13, 1-15 (2017).

[32] Vapnik, V. N. An overview of statistical learning theory. IEEE Trans. Neural Netw. 10, 988-999 (1999).

[33] Smits, G. & Jordaan, E. M. Improved SVM regression using mixtures of kernels. In Proc. 2002 International Joint Conference on Neural Networks, 2785-2790 (Honolulu, Hawaii, 2002).

[34] Breiman, L. Random forests. Mach. Learn. 45, 5-32 (2001).

[35] Lin, C. H., Chen, R. T. & Chan, Y. K. A smart content-based image retrieval system based on color and texture feature. Image Vis. Comput. 27, 658-665 (2009).

[36] Guo, W., Rage, U. K. & Ninomiya, S. Illumination invariant segmentation of vegetation for time series wheat images based on decision tree model. Comput. Electron. Agric 96, 58-66 (2013).

[37] Donis-González, I. R., Guyer, D. E. & Pease, A. Postharvest noninvasive classification of tough-fibrous asparagus using computed tomography images. Postharvest Biol. Technol. 121, 27-35 (2016).

[38] Xiong, X. et al. A high-throughput stereo-imaging system for quantifying rape leaf traits during the seedling stage. Plant Methods 13, 1-17 (2017).

[39] Hämmerle, M. & Höfle, B. Direct derivation of maize plant and crop height from low-cost time-of-flight camera measurements. Plant Methods 12, 1-13 (2016).

[40] Andújar, D., Ribeiro, A., Fernández-quintanilla, C. & Dorado, J. Using depth cameras to extract structural parameters to assess the growth state and yield of cauliflower crops. Comput. Electron. Agric 122, 67-73 (2016).

[41] Liu, M., Tuzel, O., Ramalingam, S. & Chellappa, R. Entropy rate superpixel segmentation. In Proc. 2011 IEEE Conference on Computer Vision and Pattern Recognition, 2097-2104 (Providence, Rhode Island, 2011).

[42] Yang, S. et al. Method for measurement of vegetable seedlings height based on RGB-D camera. Trans. Chin. Soc. Agric. Machinery. 50, 128-135 (2019).

[43] Chen, D. et al. Dissecting the phenotypic components of crop plant growth and drought responses based on high-throughput image analysis. Plant Cell Online 26, 4636-4655 (2014).

[44] Golzarian, M. R., Frick, R. A., Rajendran, K., Berger, B. & Lun, D. S. Accurate inference of shoot biomass from high-throughput images of cereal plants. Plant Methods 7, 1-11 (2011).

[45] Mortensen, A. K., Bender, A., Whelan, B. & Barbour, M. M. Segmentation of lettuce in coloured 3D point clouds for fresh weight estimation. Comput. Electron. Agric. 154, 373-381 (2018).

[46] Tudela, J. A., Hernández, N., Pérez-Vicente, A. & Gil, M. I. Postharvest biology and technology growing season climates affect quality of fresh-cut lettuce. Postharvest Biol. Technol. 123, 60-68 (2017).

[47] Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436-444 (2015).