Development of real-time onion disease monitoring system using image acquisition

Abstract

In this study, real-time disease monitoring was conducted on onion which is the most representative crop in Republic of Korea, using an image acquisition system newly developed for the mobile measurement of phenotype. The purpose of this study was to improve the accuracy of prediction of disease and state variables by processing images acquired from monitoring. The image acquisition system was consisted of two parts, a motorized driving system and a PTZ (pan, tilt and zoom) camera to take images of the plants. The acquired images were processed as follows. Noise was removed through an image filter and RGB (red, green and blue) colors were converted to HSV (hue, saturation and value), which enabled thresholding of areas with different colors and properties for image binarization by comparing the color of onion leaf with ambient areas. Four objects with the most significant browning in the onion leaf to the naked eye were selected as the samples for data acquired. The thresholding method with image processing was found to be superior to the naked eye in identifying accurate disease areas. In addition, it was found that the incidence of disease was different in each disease area ratio. As a result, the use of image acquisition system in image processing analysis will enable more prompt detection of any changes in the onion and monitoring of disease outbreaks during the crop lifecycle.

Keywords

imaging acquisition system / disease / downy mildew / onion

Introduction

Onion is one of the most frequently used seasoning vegetables around the world with an area of cultivation that is continuing to increase. As onion has become known as a healthy food worldwide, the area of cultivation in Republic of Korea has increased by 10% in 2016 to nearly 20 × 10³ hm² with an 18.7% increase of production to 1.30 Mt^[1]. However, damage to onion has increased gradually due to global warming and drought stress from climate change. Downy mildew, which is the severest onion disease, is caused by the soilborne pathogen, Peronospora destructor. The disease causes a serious reduction in crop quantity mostly by damage to the leaves^[2]. Downy mildew occurs when there is a lack of carbohydrates and potassium in the leaves, which causes browning and death of leaf tips and roots which leads to reduced resistance and various type of damage. Downy mildew does not occur when the temperature of the soil is 15°C or below, but outbreaks are most common when the soil temperature is 25–28°C^[3]. Rain before harvest or wet packaging can also cause infection. In addition, onions are quickly infected during the repetitive planting from mid-April to early May. The disease develops in autumn and spring, particularly in mid-April when the cold rain falls. The infected onion leaf exhibits early chlorosis and premature death. Therefore, with the recent expansion in the cultivation of the crop, it has become necessary to introduce advanced crop management technology for improved productivity and stability of production. To meet this need, more agricultural studies are evaluating drones for disease monitoring. However, despite their usefulness in periodic measurement, drones are not the most convenient tool for real-time and long-term measurements, in a rapidly changing climate. For predicting disease development and crop production, it is necessary to extract information, such as the location and size of the crop, through continuous monitoring. Accordingly, a real-time imaging device is required for measuring the growth and diseases of the crop connected to information technology. The current research on onion cultivation using a real-time imaging device is limited to only a few researchers. Min et al.^[4] carried out study to develop the vision system which automatically finds out an optimum transferring period of plants (Perilla, Platycodon grandifloums and Lactuca sativa) by using image processing. Miller^[5] showed that field monitoring of powdery mildew epidemics on strawberry revealed patterns closely allied with host phenology. Chung et al.^[6] also established a system to predict the time for insect control by assessing the flowering time of magnolia, which can be grown adjacent to onion crops. Son et al.^[7] showed that study was conducted to develop the vision system of a robotic tranplanter for plug-seedling. Yoon et al.^[8] developed an image processing system that classifies tomato, cabbage and broccoli using the characteristics of RGB (red, green and blue) colors and estimates germination percentage. Na et al.^[9] showed that suggests the algorithm that removing the shadow of vehicle using the characteristic of the HSV color model, and the algorithm that removing disconnected component using connected component analysis for removing outline process in field of post process. Mo et al. ^[10] showed that performed image processing using a digital camera to diagnose and evaluate the infection of powdery mildew using the color information in cucumber. Min et al.^[11] showed that conducted a study on a shape recognition algorithm to recognize the correct shape and position of cucumber, and analyzed the BP (backpropagation) and associative memory of an imaging device.

As a basic study for the real-time prediction of the outbreak of the downy mildew of onion, this study focused on real-time monitoring of the change in onion leaf color using an image processing system. A system was developed to measure color change using a PTZ (pan, tilt and zoom) camera. In addition, an image processing algorithm was developed to recognize leaf browning, and the brown area was calculated against the total area to predict the progress of downy mildew.

Materials and methods

Image acquisition system

The image acquisition system consisted of a motorized driving system and a PTZ camera to take images of the plants. The driving system consists of a stepping motor and a motor control program that enabled the camera to move in upward and downward directions. The driving computer was connected to the monitoring computer through the image input computer and RS-232 serial communication port. The motor driving program was designed such that the driving computer received the signal from the image processing section and rotates the motor for control. The driving system was designed to rotate the motor in a suitable time and angle for accurate measurement of plants. Figure 1 shows the driving motor on the stand.

The image capture device (HDWC-322MIR PTZ camera, Honeywell, Morris Plains, NJ, USA) was designed to obtain the image of the target plant and two computers were used for monitoring and camera control. The PTZ camera had a magnifying lens that zoomed from 32 to 512 times for a close observation of the growth environment of the plants. Table 1 gives the specifications of the camera.

Due to the distance between the onion field and the image device, the image acquisition system was installed with the camera on top of a 1.7 m high structure such that a wider view was possible. For the operation of the motor, limit sensors were attached to the upper limit and lower limits of the stage. The operating distance of the motor was 550 mm, and an electronically controlled brake was installed. For accurate measurements, the motor was equipped with a driving system that rotated the motor 360° for a total of 15 times at 24° increments. To facilitate image acquisition during snowy or rainy weather, the system was installed under a cover. In preparation for heavy snow or rain, the motor is designed to stop with a signal automatically coming from a contact switch in a rain sensor. Figure 2 shows the camera as installed.

Image processing algorithm

The image processing algorithm performed the following steps sequentially: image noise filtering, HSV (hue, saturation and value) conversion, color distance calculation, detection by distance threshold, detection by blob detection and area rate calculation. Figure 3 shows the sequence of the image processing algorithm.

First, images of overall height of onion were acquired with the PTZ camera. To filter the noise from the image, a low-pass filter was applied to input data as one word. The HSV conversion used the RGB colors that signify the coordinates of the 3D color space as a method to display the removed image when the RGB components are added to each other. If the values of RGB are given in a range of 1 to 100, the values of HSV are obtained using Eqs. (1), 2 and 3.

(1)\mathrm{H}=\{\begin{array}{l}\mathrm{60\times }{\displaystyle \frac{\mathrm{G-B}}{\mathrm{MAX-MIN}}}\mathrm{+0,}\hspace{0.17em}\mathrm{MAX=R}\\ \mathrm{60\times }{\displaystyle \frac{\mathrm{B-R}}{\mathrm{MAX-MIN}}}\mathrm{+120,}\hspace{0.17em}\mathrm{MAX=G}\\ \mathrm{60\times }{\displaystyle \frac{\mathrm{R}\mathrm{-G}}{\mathrm{MAX-MIN}}}\mathrm{+240,}\hspace{0.17em}\mathrm{MAX=B}\end{array}

(2)\mathrm{S=}{\displaystyle \frac{\mathrm{MAX-MIN}}{\mathrm{MAX}}}

(3)\mathrm{V=MAX}

where MAX and MIN are the maximum and minimum values of the RGB. HSV is constructed with a model that emulates the method that humans employ to recognize an image. As shown in Fig. 4, hue represents the color of red, blue and yellow with a range of 0–360°. Saturation, on the horizontal axis, represents the sharpness of the color, and has a range of 0% to 100%. As the saturation approaches zero, it approaches the central axis. In contrast, as saturation approaches 100, it moves away from the central axis. The value (V) represents the brightness of the color, ranging from 0% to 100%. V is presented on the vertical axis and gets closer to zero when V decreases and gets closer to 100% when it increases.

In the color distance calculation, the filtered image is obtained as the value of RGB pixel, which is subsequently converted to HSV, and the distance from random color is obtained to measure the distance between the color of each pixel and specific color. Regarding detection using distance threshold, the images were categorized as black, if below the threshold, and yellow, if they exceeded the threshold, using the thresholding method that visually inspects the image histogram that is converted to HSV. To identify the effect of image binarization using the thresholding method, each component image was used to generate image binarization using the trial and error method on the individual RGB and HSV elements as well as Otsu and ISODATA methods of automatic thresholding. The Otsu method uses the variance value to find the most appropriate value for dividing the image into two parts. For blob detection, the measured data were processed with a thresholding method. Then, the brown part of the onion leaf was detected through stain detection. For disease area rate calculation, using the naked eye, the image binarization was compared with the existing images through detected data to confirm that the downy mildew was in fact detected. Then, the disease area for each onion leaf was calculated.

Experimental method

This study was conducted in an onion field (Fig. 5) at the Bio Energy Crop Center, National Institute of Crop Science, Muan, Jeongnam, Republic of Korea. The reason for the selection is that the onion is the best growing crop of the leaf and the disease occurrence compared to other crops. The study was conducted on onion that had been planted in early November.

The image acquisition system was installed 10 m away from the field so it could measure any damage or change in the onions. Measurements were taken from 7 am to 7 pm between 20 April 2017 and 1 May 2017. Fifteen onion plants from different locations were measured at 30-min intervals. To determine the disease area, measurements were taken in an environment conducive to downy mildew over the entire test period. The measurements were conducted from mid-April which is the first infection cycle of downy mildew. Four samples were randomly selected for color image processing when the soil temperature was 25–28°C.

Results and discussion

Four representative images acquired from the PTZ camera image acquisition system are given in Fig. 6. The red circled areas indicated that the leaves are diseased.

Processed versions of the images in Fig. 6 used to determine disease area are given in Fig. 7. The detected disease was 5.34% (Fig. 7a), 6.94% (Fig. 7b), 3.38% (Fig. 7c) and 4.40% (Fig. 7d). Thus, it was concluded that the sample was at an early stage of downy mildew infection. Table 2 shows the detection rate and removal rate of the browned area on the onion leaf.

Given the difficulty of correctly estimating the leaf area affected by downy mildew in onion with the naked eye, it was necessary to use image binarization via the numerical thresholding method to accurately determine the disease area. However, this method erroneously included the soil in the onion field instead of only extracting the brown part of the leaf during image processing. This was corrected with the image acquisition system device developed in this study to continuously predict the change in onion growth and disease progression. Thus, the use of growth information that is measured continuously and image processing, could help in the provision of an optimum environment for each growth cycle. However, it is necessary to ensure that the image processing system does not erroneously included the surrounding elements rather than just the onion leaf.

Conclusions

This study measured disease area in onions by developing an image acquisition system and processing the obtained images. The image acquisition system acquired sequential images of the onion field at a 30-min time interval. Four randomly selected samples were analyzed using an image processing algorithm. The result of image processing showed that browning occurred on the leaf of onion. The disease areas ranged from 3.38% to 6.94%. However, the accuracy of these determinations should be analyzed in the future to ensure that only the brown part of the leaf, and not the surrounding soil, is recognized. The image acquisition system can be used to sense the color, height and disease of leaves of the onion. Therefore, it is possible to checked the state change of the crop by using the image acquisition system. In addition, for more practical and effective monitoring, additional analysis of the images will be required to identify reactions related to growth conditions and climate.

References

[1] Lim E C, Kim J J, Chang S S, Shim C B, Lee J H. Implementation of a multimedia pest prediction management system for mobile users. Journal of the Korea Contents Society, 2007, 5(1): 1–5

[2] Kim G Y, Ryu K H, Chun S P. Identification of crop growth Stage by image processing for greenhouse automation. Journal of The Korean Society for Agricultural Machinery, 1999, 24(1): 25–30

[3] Kim G Y, Ryu K H, Chae H Y. Analysis of water stress of greenhouse crops using infrared thermography. Journal of The Korean Society for Agricultural Machinery, 1999, 24(5): 439–444

[4] Min B R, Kim W, Kim D W, Lee D W. Determination of transferring period of several plants using image processing. Journal of Bio-Environment Control, 2004, 13(3): 178–184

[5] Miller C T. Studies in the control of the powdery mildew disease of strawberry and of the biology of the fungal pathogen Sphaerotheca macularis f. sp. Fragariae. Dissertation for the Master Degree.California: University of California, 2000

[6] Chung S O, Kim Y J, Lee K H, Sung N S, Lee C H, Noh H K. Remote monitoring of light environment using web-camera for protected chrysanthemum production. CNU Journal of Agricultural Science, 2015, 42(4): 447–453

[7] Son J R, Kang C H, Han K S, Jung S R, Kwon K Y. Recognition of missing and bad seedlings via color image processing. Journal of the Korean Society for Agricultural Machinery, 2001, 26(3): 253–262

[8] Yoon D H, Oh S Y, Nam K W, Eom K C, Jung P K. Changes of cultivation areas and major disease for spicy vegetables by the change of meteorological factors. Journal of Climate Change Research, 2014, 5(1): 47–59

[9] Na I S, Kim S H, You S M, Lee K S. Removal of the shadow of vehicle images using HSV color model. Journal of Korea Transportation Research Society, 2008, 58(1): 1065–1070

[10] Mo C Y, Lee H J, Ko S J, Yang K Y. Control efficacy of several fungicides against downy mildew of onion at nursery seedling stage. Journal of the Korean Society of Plant Pathology, 2016, 22(3): 208–212

[11] Min B R, Kim W, Kim D W, Seo K W, Lee C W, Lee D W. Analysis of the back propagation and associative memory for image system. Journal of Proceeding of Bio-Environment Control, 2003, 12(1): 181–190

RIGHTS & PERMISSIONS

The Author(s) 2018. Published by Higher Education Press. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0)