Introduction
It is projected that the human population will reach 9 billion by 2050. This increase requires a 3.3% annual growth in crop production, while the current annual growth rate is only 1.3%
[1]. Accelerated crop improvement is needed to satisfy the global food demand
[2] and counter the increasingly worsening abiotic and biotic stress conditions for plants
[3]. Advances in genetic technologies provide crop breeders with the potential to develop high-yielding, stress-tolerant plants through genetic selection
[4]. However, the lack of access to phenotypic data hinders this effort. Plant breeders and farmers have been making cultivar selections based on phenotypes long before the discovery of DNA and molecular markers
[5]. They have learned that superior genetic variations can only be identified through a large number of crosses between breeds and cultivars under various environmental conditions. Advances in high-throughput phenotyping (HTP) offer much faster and less expensive genomic information, and pave the way for the development of large mapping populations and diverse panels of thousands of recombinant inbred lines
[6]. Existing phenotyping technologies, in contrast, simply cannot provide sufficiently fast and accurate measurements of the traits of a sufficient number of lines and have, thus, become the bottleneck for crop breeding and development. Current phenotyping technologies can be characterized as slow, expensive, labor-intensive, and often destructive
[7].
Plant performance is affected by the nature of the genome and the characteristics of the surrounding environment on growth, quality and productivity, such as temperature, light, weather events, and soil characteristics, along with plant disease, weeds, and insect pests. A large number of recent studies have shown that these factors can be effectively monitored using various imaging technologies
[8]. Examples include applications of fluorescence sensing for quantitative analysis of photosynthesis, visible imaging for shoot biomass estimation, visible-near infrared spectroscopy for identifying physiological changes induced by nutrient deficiencies, and thermal imaging for detecting water stress
[9].
Phenotyping platforms
Over the last few years, various types of ground-based HTP platforms, including stationary, vehicle-based, self-propelled, and portable, have been developed. The stationary phenotyping platforms run on cranes, towers or cables (similar to stadium cameras). The vehicle-based phenotyping devices are mounted on a tractor or pulled by a vehicle
[10]. Self-propelled phenotyping platforms are specifically designed to carry sensors and data acquisition systems. Some of the self-propelled phenotyping machines are robotic, with automatic drive/steering mechanisms or automatic guidance systems. The portable phenotyping platforms are carried by human operators or moved on wheeled carts
[11,12]. Some HTP platforms are fully automated facilities consisting of robots, precise environment control, and sensing equipment.
Aerial phenotyping uses satellites, low-altitude aerial vehicles and unmanned aerial vehicles. Both ground-based and aerial phenotyping platforms can effectively assess overall growth and development of the plants, although each type has its disadvantages and limitations. For example, stationary phenotyping platforms often have limited area of coverage, tractor-mounted phenotyping platforms rely on experienced operators, robotic pheno-typing platforms are expensive and often have safety concerns, and portable phenotyping platforms are generally less efficient. The main concerns for the aerial phenotyping systems are limited resolution and available payload.
Early HTP platforms were mainly used to measure the traits of individual plants in a controlled environment. However, some important plant traits are best expressed when they are grown in populations under field conditions. Therefore, efforts have been intensified to develop field-based HTP platforms
[13]. The earliest platforms carried a single type of sensor. For example, Sui et al.
[14] used a three-wheeled cart with multiple ultra-sonic sensors around a single row of a crop to scan the plant canopy. Montes et al.
[15] used tractor- and harvester-mounted reflectance sensors to collect spectral data. More recently, McCarthy et al.
[16] used a machine vision system to measure internode length in cotton. However, the major limitation of these systems was their inability to measure multiple traits, which is often needed in crop breeding experiments.
To measure multiple traits, Lan et al.
[17] devised a system that uses integrated sensors and multispectral/hyperspectral imaging devices to measure crop canopy height, leaf area index (LAI), and normalized difference vegetation index (NDVI). Crain et al.
[12] developed a pushcart-type phenotyping platform that used an infrared thermometer, an ultrasonic sensor, a digital camera, and a portable GPS to measure georeferenced leaf temperature, crop height and other morphological traits. Comar et al.
[18] used a platform with multiple instruments, including a passive spectrometer, an irradiance probe, a GPS unit and a digital camera, to monitor micro-plots of wheat cultivars in field conditions. Similarly, White and Conley
[19] used a handcart to position multiple sensors over two rows. Although these platforms provided opportunities for fast acquisition of phenotyping data, they required continuous labor to move the carts in the field. Furthermore, these platforms were incapable of providing truly “high-throughput” phenotyping data for modern varietal development programs. To improve the throughput, Andrade-Sanchez et al.
[20] used a high-clearance vehicle to carry four sets of sensors to measure canopy height, reflectance, and temperature simultaneously on four adjacent rows of cotton at a rate of 0.84 ha·h
−1. A similar HTP platform was developed by Barker et al.
[11] to collect multiple crop trait data at a high throughput. Low-altitude, manned and unmanned aerial vehicles have also been used for phenotyping in recent research
[20], but aerial vehicles are usually weight limited and cost is another concern.
Plant phenotypical traits and their significance
Crop traits measured by phenotypers can be classified into three categories: morphological, physiological, and pathological. Morphological traits are direct measures of the forms and structures of plant organs and their structural features, such as shape, color (spectral reflectance), texture, pattern and size. Physiological traits are direct measures of the functions of plant organs, such as photosynthesis, respiration, nutrition, hormone functions, stress resistance and plant water relations (stomata function and transpiration). Pathological traits are direct measures of plant diseases caused by pathogens (infectious organisms) and environmental conditions, including fungi, bacteria, viruses, nematodes, insects and parasitic plants. Due to difficulties in direct measurement, physiological and pathological traits are often indirectly assessed through morphological measurement. For example, the photosynthesis function of a plant, as a measure of plant stress, may be indirectly estimated by measuring chlorophyll fluorescence emission, because light energy absorbed by a leaf is used to drive photosynthesis. More chlorophyll fluorescence emission indicates less photosynthesis.
Morphological traits
Leaf area and fruit color are typical examples of morphological traits. Given that all green surfaces of crops, such as spike, stem, sheath, and leaf lamina, are related to the above-ground biomass, surfaces for transpiration/water loss, photosynthetic potential, and light interception
[21], leaf measurements are important for studies of plant growth. LAI is a trait relating to leaf area that indicates the interception of radiation by the canopy and reflects the photosynthetic activity of the crop canopy
[22]. LAI also is a useful feature reflecting plant transpiration and carbon dioxide exchange, providing quantitative structured information on the energy exchange between the plants and their external environment
[23].
Changes in stem and leaf colors at different growth stages often are important indicators of crop growth status. Nutritional status, population structure, and diseases affect the growth and development of crops, causing color changes in different parts of the plant. Thus, using color recognition combined with a variety of classification schemes, many phenotypic parameters involving colors can be established.
Plant roots usually are complex, three-dimensional systems that contain many standard traits (such as root length density, number of roots, root size, total root surface, root length, root growth response, average root radius, average root length, number of lateral roots), and several newly defined traits, such as network perimeter, solidity, and convex area
[24].
Physiological traits
Examples of plant physiological traits include canopy temperature, leaf water potential, stomatal conductance, and chlorophyll content. Water metabolism is an important physiological process of crops. It has an important influence on the growth, development, yield, and quality of crops. Through transpiration, crops lose water, which takes away the heat, thus regulating the plant body temperature (canopy temperature). Therefore, canopy temperature is closely related to the water status of crops and can affect water use efficiency, stomatal conductance, photosynthesis activity, and transpiration rate
[13]. Mea-suring canopy temperature is one of the most important methods used to monitor crop water conditions.
Leaf water potential is a trait related to plant water status. Water in plants is transported within the xylem system under negative pressure which is positively related by the degree of water stress
[25]. Stomatal conductance is a trait related to leaf and canopy gas exchange. It responds rapidly to soil water status and controls photosynthetic activity and growth
[9]. Higher density, larger size, and a greater degree of opening of the stomata allow increased conductance and potentially higher photosynthesis and transpiration rates. Stomatal conductance is important for breeding because it is closely related to heritability and yield. Greater leaf conductance under warm temperatures has been associated with cooler canopy temperatures
[13].
Leaf chlorophyll content has a positive relationship with dry root biomass, seed yield, and tolerance to iron-deficiency chlorosis. Nodulation and nitrogen fixation status can also be identified by leaf chlorophyll measurement
[13].
Pathological traits
Like humans, plants can be infected by diseases. Plant disease is a major threat to global agriculture, accounting for at least 10% reduction in global yields
[26]. Plant diseases affect the growth and development of crops, limiting crop yields and making agricultural products unfit for consumption
[27].
Plant roots are critical for plant growth and development. Indispensable aspects of a plant’s life, such as water and nutrients, are absorbed by the roots. At the same time, plant roots store carbohydrates and other substances, and enable biosynthesis of hormones which are required for growth
[28]. Therefore, the impact of diseases on the root structure may affect crop yields
[29].
Imaging systems for plant phenotyping
Various types of imaging systems provide important sensors for plant phenotyping. Morphological traits of plants, including colors of seeds, leaves, canopies, fruits, and roots, shapes and sizes of seeds, sizes, numbers, areas, textures, angles, architectures, and total volumes of canopies, leaves, and roots, and volumes sizes, shapes, numbers, and spatial distributions of fruits, can be directly measured using two- or three-dimensional imaging systems.
The physiological characteristics related to plant organ functions and the pathological features related to pests and diseases may be directly or indirectly measured by using visible light, thermal, fluorescence, spectral, and three-dimensional imaging technologies. Araus and Cairns
[5] give examples of images taken with different imaging systems for crop monitoring.
Visible-band imaging
Visible-band imaging systems are the fundamental tools for measuring morphological traits (color, shape, size, and texture) of plants. As the technology advances, digital red-green-blue (RGB) cameras that are suitable for aerial and ground applications have become available at affordable prices. With their speed and high quality under low light conditions, digital single lens reflex (SLR) cameras have become the first choice for many phenotyping applications.
Fluorescence imaging
Fluorescence is the light emitted from a plant when the plant absorbs radiation in a shorter wavelength. The typical fluorescent part of a plant is the chlorophyll complex
[30]. Irradiating the chloroplasts with blue or actinic light will result in re-emission of a portion of the absorbed light by the chlorophyll, which is the fluorescence. The proportion of the re-emitted light in the absorbed radiation is positively correlated with the ability of the plant to metabolize the harvested light. Thus, chlorophyll fluorescence is a good indicator of the ability of plants to absorb actinic light. Moreover, the combination of an actinic light source with brief, saturating blue pulses may be used to estimate the efficiency of photo-assimilation, non-photochemical quenching and other physiological parameters of plants. The theoretical basis of fluorescence imaging is that the intensity of the fluorescence signal emitted after the excitation of the fluorescent substance is linearly related to the amount of fluorescence within a certain range. In short, fluorescence imaging is the imaging of fluorescent signals. The fluorescence imaging system includes a fluorescence signal excitation system (excitation light source and optical transmission component), a fluorescence signal collection component, and a signal detection and amplification system
[31].
The principle of multicolor fluorescence imaging is that UV illumination (ranging from 340 to 360 nm) produces two types of fluorescence, in the red-infrared and blue-green spectral regions, respectively
[31]. This technique allows simultaneous capture of fluorescence signals in four bands—blue (440 nm), green (520 nm), red (690 nm) and infrared (740 nm). The emission of fluorescence signals within the blue-green region (with maxima near 440 and 520 nm) is an indicator of the cinnamic acids (primarily ferulic acid) that is present mostly in the cell walls; and the emission within the red-infrared region (with maxima near 690 and 740 nm) is a good indicator of chlorophyll α molecules in the antenna and reaction center of the photosynthetic photosystem II (PSII) from chloroplasts in the mesophyll cells. The leaves tested for chlorophyll fluorescence are also used to measure chlorophyll content
[13]. Changes in the fluorescence emission and the ratios between emissions in different regions, such as F440/F690 and F440/F740, may be used as indicators of stress, and the F690/F740 ratio has been shown to be a good indicator of the chloroplast content
[32].
Thermal imaging
Thermal imaging allows for visualization of infrared radiation, giving indications on temperature distribution across an object’s surface. The sensitive spectral range of thermal cameras is 3–14 µm, and the most commonly used wave bands for thermal imaging are 3–5 or 7–14 µm, respectively. Within these two wave bands, transmission of infrared radiation through the atmosphere is close to its maximum value. The thermal sensitivity in the 3–5 µm band is higher than that in the 7–14 µm band because shorter wavelengths correspond to higher energy levels. However, the use of longer wavelengths may be advantageous for certain applications. For example, for targets at a long distance, wavelengths between 8 and 14 µm would minimize errors caused by atmospheric absorption of infrared radiation
[33,34]. Thermal imaging is a practical alternative to measurement of various plant traits. Thermal measurement of plants mostly relies on evaporation, with high and low levels of temperature reflecting closing and opening of stomatal structure, respectively. In recent years, with improved thermal sensitivity, reduced price, more friendly user interface, and enhanced resolution, thermal imaging has found many applications in plant phenotyping
[34].
Spectral imaging
Multispectral and hyperspectral imaging technologies have become powerful tools for plant sensing and monitoring. These technologies provide both spatial and spectral information on the plants. The range of wavelength in the spectral domain can be from 400 to 2500 nm. In the visible spectral range (400–700 nm), the reflectance of leaves or canopies is particularly low due to the absorption of light by leaf pigments, especially chlorophyll. As the wavelength increases to the near infrared band, the reflectivity increases dramatically
[32]. In the near infrared (700–1200 nm) range, scattering within the leaves reflects a large amount of incident radiation. As the wavelength gradually increases to 2500 nm, the reflectivity gradually decreases, due to the absorption by water present in the leaves.
Other imaging technologies
Stereo imaging
Commercially available image sensing technologies for 3D mapping of plants include light detection and ranging (LiDAR; or laser scanner) sensors, stereo vision, ground penetration radar, photon mixer devices, time-of-flight (ToF) cameras, and devices designed for consumer-gaming interface, such as Microsoft Kinect
[32].
LiDAR is a remote sensing technology that measures the distance to a target by illuminating the target with a pulsed laser light and measuring the reflected pulses. By scanning through the measured target, LiDAR can create digital 3D representations of the target. LiDAR devices can be used to acquire multi-source phenotypic data during the entire crop growing period and extract important plant morphological traits, such as plant height, plant width, leaf area, leaf length, leaf width, and leaf inclination angle for plant biological and genomic studies
[32].
Ground-penetrating radar uses pulses of high-frequency radio waves to image subsurface structures based on differences in dielectric constant
[13]. In addition, cross-borehole GPR has been shown to be a valuable means of assessing the presence and amount of soil water within the field of hydrogen physics
[35].
Stereovision is an important subject in the field of computer vision for reconstructing the three-dimensional geometry of scenes. Stereo vision uses two or more cameras to study 3D structures and their motions
[32].
The ToF detects the distance by measuring the round trip time of light pulses from the sensor to the target. This technology is similar to the 3D laser sensor principle, except that the 3D laser sensor provides only point-by-point scans, and the ToF camera gives the depths of all pixels within the entire image.
Tomographic imaging
Magnetic resonance imaging (MRI), positron-emission tomography (PET), and computed tomography scanning (CT) are three important technologies for tomographic imaging. MRI scanners use strong magnetic fields, electric field gradients, and radio waves to generate images of the organs
[32]. The PET generates changing three-dimensional images by detecting a pair of gamma rays emitted by a positron-emitting tracer introduced into the plant. CT uses an accurately collimated beam of X-rays to scan one section of an object at a time with a highly sensitive detector and reconstructs the object in 3D. In recent years, these technologies have been tested for plant phenotyping, although there are few examples found in the literature.
Applications of imaging systems in plant phenotyping
Various imaging technologies have been applied in plant phenotyping. Applications found in recent literature are summarized in this section.
Visible light imaging
Visible light imaging has been widely used in the measurement of various aspects of plant structure for its low price and ease of maintenance. In a controlled environment such as a growth chamber or a greenhouse, visible light imaging has been primarily adopted for analyzing the yield traits, shoot biomass, biomass at anthesis, imbibition and germination rates, panicle traits, coleoptile length, leaf morphology, seedling vigor, seed morphology, leaf area, and root architecture
[13]. Applications of visible light imaging in measuring different types of plant traits are reviewed in the following sections.
Morphological traits
For many horticultural crops, the color of fruits obtained by visible light imaging often can serve as an effective indicator of the maturity, flavor, and freshness of the fruits. A number of breeding programs have conducted research to understand the genetic and physiological mechanisms that lead to fruit color. Computer-based analysis of objects from digital images provides high-quality data and characterization
[13].
Darrigues et al.
[36] developed a new software tool, tomato analyzer color test (TACT), to analyze the genotypic differences associated with color parameters. Their research shows that TACT is applicable to other fruits and vegetables of various colors, and can accurately capture and describe the characteristic color of each crop. Similarly, Gonzalo et al.
[37] used the Tomato Analyzer program to analyze the shape attributes of tomato.
Various monitoring platforms using visible light imaging technology have been developed to observe the distribution of plant roots of different genotypes. For example, Anjalis's gel-based growth platform enabled non-destructive detection and real-time observation of the root structure by rotating the platform to different angles
[29].
Physiological traits
Wang et al.
[38] used visible light imaging to analyze the sugar content of citrus fruits. They found a clear correlation between the sugar content and certain parameters of the color image. They also found that smaller citrus fruits usually have higher sugar contents. Zakaluk and Ranjan
[39] conducted studies using RGB reflectivity to predict the leaf water potential of potato plants. They found that the G and R image bands can be used to separate shadows, soil, senescent leaves, green leaves, and flowers. They also found an inverse linear relationship between the green image band and soil nitrogen content.
Golzarian et al.
[40] presented a method for accurate estimation of plant shoot dry weight from two-dimensional images. This method enables high throughput, nondestructive estimation of biomass for cereal plants under salt stress.
Pathological traits
Wang and Li
[41] used visible light imaging to achieve non-destructive detection of external defects (green, germinated and diseased areas) of potato. The green areas were detected using multicolor models of RGB, HSV and LABORATORY. For germination and disease, they used the Laplace Operator Gray Variance method.
Mutka and Bart
[27] used visible light imaging technology to study the Xanthomonas in a leaf spot caused by citrus ulcer disease on grapefruit leaves. They found that automated image analysis was more reproducible than visual assessment in multiple measurements, and image analysis enhanced the ability to distinguish genotypes with varying degrees of disease severity. Raza
[42] reported an algorithm for registration of thermal and visible light images of diseased plants based on silhouette extraction with high accuracy. The combination of visible light and thermal imaging technologies was found to be helpful in early discovery of diseases in plants.
Fluorescence imaging
Fluorescence imaging can be used to characterize plant health and photosynthetic activities related to physiological and pathological traits. Fluorescence imaging also can be used to assess the respiration function of plants, to detect effects of plant insect- and disease-resistance genes, and to monitor plant pathogens. In addition, fluorescence imaging can be used to diagnose plant responses to abiotic and biotic stresses, such as drought and salinity, prior to growth recession
[43]. Applications of fluorescence imaging in measuring plant physiological and pathological traits found in the literature are summarized below.
Physiological traits
Fluorescence imaging is a very sensitive detection system that can provide rapid assessment of plant physiological status under a variety of situations. In crop improvement programs, fluorescence imaging is often used for the detection and screening of plant resistance genes. For example, Swarbrick et al.
[44] used quantitative imaging of chlorophyll fluorescence to study the resistive response of barley leaves against
Blumeria graminis. Chaerle et al.
[45] screened sugar beet lines that differed in susceptibility to
Cercospora beticola infection using chlorophyll fluorescence imaging and found that differences in fluorescence intensity were measurable between susceptible and resistant plants.
Pathological traits
Fluorescence imaging can be a direct measure of the physiological characteristics of plant photosynthesis and related metabolic disorders, and an indirect measure of the pathogenical/physiological characteristics such as herbicidal effects, stomatal heterogeneity, disease resistance, and resistance to other biotic and abiotic stresses. Barbagallo et al.
[46] used changes in parameters Fv/Fm in fluorescence images to detect growth of seedlings and herbicide-induced metabolic perturbations. Bürling et al.
[47] found that by using pulse amplitude modulated (PAM) chlorophyll fluorescence imaging technology, the level of resistance of wheat cultivars to the leaf rust (
Puccinia triticina Erics.) pathogen can be evaluated.
Thermal imaging
Thermal imaging can be used to measure plant canopy temperature, thus monitoring leaf stomatal opening and plant transpiration intensity. This technique can also be used to analyze the leaf water potential of plants to monitor plant water energy. In addition, it can be used to monitor the genetic variations of plants and detect pathogens (bacteria, fungi, and viruses). Applications of thermal imaging in measuring various types of plant traits are summarized below.
Morphological traits
Thermal imaging technology has been used to monitor the health state of fruit trees, to study their three-dimensional structures, and to identify fruits. Narváez et al.
[48] used ground-based thermal imaging technology to extract three-dimensional features of fruit trees in an orchard. The experiments showed that this technology is capable of determining heat-related morphological characteristics of the trees. Xu and Ying
[49] used infrared thermal imaging technology based on the fact that different parts of the fruit tree have different heat radiation characteristics, and combined the thermal image with a grayscale image. With the help of edge detection operators, they were able to successfully recognize the fruits using temperature differences.
Physiological traits
Infrared thermography is a non-invasive technique which does not cause damages to the objects to be measured. Thermal imaging systems can provide information on the spatial distribution of surface temperature of an object and the temporal changes of this spatial distribution if the measurement is repeated over a period of time.
Cohen et al.
[50] used a radiation infrared camera to take thermal images to estimate the moisture status of cotton. Their result showed that leaf water potential can be well estimated by combining thermal images with spatial analysis. In another study, thermal canopy imaging of vines was analyzed to study stress responses under different watering regimes
[51]. The International Maize and Wheat Improvement Center (CIMMYT)
[52] performed a high-throughput thermal stress analysis of maize phenotypic adaptation to water stress using thermal imaging technology. The main goal was to evaluate corn canopy temperature and crop water stress index (CWSI) and to determine the role of genotypic variation in corn moisture traits. The results indicated that infrared thermography is potentially useful as a high-throughput screening method for drought resistant breeding programs
[53].
Pathological traits
Invasions of pathogens may cause closures of stomata in plants, which may in turn lead to a decrease in transpiration rate and changes in temperature of the plant body. This phenomenon has a strong influence on the infected parts of plants and can be located and monitored by thermal imaging technology. In 2013, Calderón et al.
[54] used aerial thermal and spectral imaging technologies to monitor
Verticillium wilt caused by
Verticillium dahliae, a soil-borne fungus, in traditional olive growing areas for three consecutive years. They found that, when the olive trees were infected with
Verticillium wilt at an early stage, as the degree of disease intensified, the temperature of the crown increased significantly and the stomatal conductance of the leaves was significantly reduced, demonstrating the possibility of using thermal imaging to monitor the early stages of disease in trees.
In 2015, Dumont et al.
[55] used thermal imaging to detect the seed quality of Norway spruce and identified active, empty, disease-causing seeds using support vector machines and sparse logistic regression based on feature selection. In 2017, Jafari et al.
[56] used visible light and infrared thermal images to detect the preclinical characteristics of roses infected with powdery mildew and gray mold. Thermal images were used to separate healthy from susceptible plants.
Spectral imaging
Spectral imaging technology can be used to quickly and efficiently analyze the plant’s morphological, physiological, and pathological characteristics without causing damages to the plants. Examples include detection of canopy chlorophyll content, leaf and canopy senescence, and plant water status, identification of crop types, detection of plant health status and analysis of plant resistance to pests and disease. Applications of spectral imaging in measuring different types of plant traits found in recent literature are summarized below.
Morphological traits
Borregaard et al.
[57] used an imaging spectrometer to identify potato, sugar beet, and three kinds of weeds under artificial lighting conditions. Through acquisition, analysis, and comparison of images taken within different spectral bands, the crops and weeds could be effectively distinguished. In addition, Mao et al.
[58] used Fourier transform infrared (FTIR) spectroscopy to obtain the spectral reflectance of several crops, including wheat and amaranth. Seven characteristic wavelengths for modeling were extracted to discriminate wheat against weeds. This technology can effectively provide reliable technical guidance for herbicide and fertilizer applications in precision agriculture.
Physiological traits
Multispectral and hyperspectral imaging data have been used to form various moisture indexes in the infrared absorption band to estimate canopy moisture content
[59–61]. By using wavelet transformation to enhance the data analysis, the sensitivity of canopy water content measurement is improved. In combination with liquid crystal tunable filters, multispectral imaging can be used to capture green fluorescent protein, natural red light, and near-infrared chlorophyll fluorescence, and is thus becoming a powerful tool for plant health monitoring. A multispectral plant health monitoring system developed by Abboud et al.
[62].
Pathological traits
Near-infrared reflectance spectroscopy has been used as a high-throughput screening tool in a sugarcane breeding program to study pest and disease resistance. Hyperspectral imaging and non-imaging sensors have been shown to be effective in detecting changes in plant vigor
[63]. When the biophysical and biochemical properties of plant tissues change, their emissivity changes accordingly. Therefore, the pathological characteristics of the plant can be effectively detected. Disease-specific symptoms, such as changes in optical characteristics, and typical fungal structures, including powdery mildew, rust, and downy mildew, caused by continuity of chlorotic and necrotic tissues can also be detected
[64,65].
In 2011, Moshou et al.
[66] developed a real-time remote sensing system that can be mounted on a tractor or a robotic platform. Hyper- and multi-spectral imaging systems were used to simultaneously perform on-site evaluation of crop canopies to predict plant disease in its early stages.
Hyperspectral imaging has also been used to detect head blight infections in wheat. At present, aerial and terrestrial hyperspectral imaging systems are being developed for applications in agriculture and natural environment monitoring
[27].
Other imaging techniques
Other imaging techniques used for high-throughput plant phenotyping are three-dimensional mapping/imaging and tomography. Three-dimensional mapping/imaging devices include light detection and ranging (LIDAR or laser scanner) sensors, stereo vision, photon mixer devices (PMD) and ToF cameras. The main means of tomographic imaging are MRI, PET or CT imaging. Applications of these sensors in measuring various types of plant traits are reviewed in the following sections.
Morphological traits
Takizawa et al.
[67] used stereovision to build 3D models of plants. From these models, information related to plant characterization, such as plant height, leaf area, and shape, can be obtained. Kazmi et al.
[68] changed the exposure time of a sensor to analyze and compare indoor and outdoor (with shadow and sunlight) depth images of leaves with a ToF camera, which is sensitive to ambient light. The ToF camera had a low resolution, however, under appropriate conditions, it can provide precise depth data with a high frame rate. Kraft et al.
[69] tested ToF imaging with a commercial camera, and concluded that, for large leaves of plants (such as corn and maple), the distance measurement is reasonably accurate. However, the measurement depends on the color contrast between the leaves and the background. The sunlight was not found to have a significant effect on the measurement.
Hargreaves et al.
[70] used a gel room to analyze the root characteristics of five different barley genotypes in 2D and used X-ray imaging for non-invasive measurement. Li et al.
[71] used a combination of stereovision and image processing to acquire important coordinate points from acquired images and then mapped them into a world coordinate system to complete the measurement of plant height. Omasa et al.
[72] used 3D LIDAR imaging to evaluate plant canopy height, canopy structure, carbon storage, and species characteristics at a large scale. By comparing the LIDAR data of the leaves and the canopy, the growth and morphological responses of plants were evaluated. Hosoi and Omasa
[73] used a hand-held 3D LIDAR system and a voxel-based canopy analysis to estimate the distribution of vertical canopy density structures at four growth stages of wheat: tillering, stem elongation, anthesis, and maturation. A regression analysis was conducted to extract the dry weight of each organ, which was then used as an effective indicator of plant growth status, allowing effective agricultural management.
Physiological traits
Windt et al.
[74] used a special MRI device to study the phloem and xylem transport of large potted plants. They compared dynamics of the xylem and phloem of four types of plants- poplar, castor, tomato and tobacco, and derived the parameters of velocity, area of diversion, and volume flow (flux). Jahnke et al.
[75] used MRI to evaluate the storage organs of beet (
Beta vulgaris) and
Raphanus sativus. At the same time, they used PET to measure the distribution and assimilation characteristics of the transport sector of a particular species. Han and Yi
[76] used MRI to monitor the water status of lily bulbs in the dormant state after being stored for 11 weeks and the changes in their metabolite contents to understand the release of dormancy in lily bulbs.
Pathological traits
Studies of pathological traits mainly used the visible light imaging, fluorescence imaging, thermal imaging and spectral imaging technologies, and little has been done in stereo imaging and tomographic imaging technologies.
Advantages and limitations of imaging systems in plant phenotyping
It is important to select the right equipment for specific phenotyping tasks. Thus, researchers must have a good understanding of the advantages and limitations of each imaging system to accomplish the tasks with good quality, accuracy, and efficiency at an affordable cost.
Visible light imaging
Visible light imaging has been widely accepted as the primary tool for plant phenotyping. The major advantages of visible light imaging are the high imaging rate, low cost, and simple operation. Many morphological traits of plants can be directly obtained from visible light images, whereas physiological and pathological traits can be indirectly inferred from analysis of visible light imaging. Major limitations include the dependence on illumination and difficulties in segmenting objects of interest, such as leaves and stems, from the background, especially when the background color is close to the color of the objects of interest. Moreover, shadows and occlusion of plant canopies often cause errors in image analysis. Some of these difficulties can be alleviated by improvements in image processing algorithms, however, and many such improvements have been reported in recent literature.
Fluorescence imaging
Fluorescence imaging is widely used for the measurement of physiological and pathological features of plants. A unique feature of fluorescence imaging is its ability to observe some physiological changes in plant tissues that are not visible to either a naked eye, or a visible light camera. This feature enables researchers to detect early signs of biotic or abiotic stresses and to take preventive or corrective measures against them. However, fluorescence imaging has been found not to be sensitive enough to water stress. In the past, fluorescence imaging has only been applied to small objects such as leaves and seedlings. Further development requires effective fluorescence imaging for larger shoot geometries and the capabilities of 3D reconstruction at an acceptable cost. Robustness and reproducibility also need to be improved. Because of large power requirement, fluorescence imaging has been mainly used in a controlled environment
[77].
Thermal imaging
Many thermal imaging systems offer high sensitivity, high precision, and a wide measurement range. Thermal imaging is a non-contact measurement tool and its operation can be easily automated for real-time measurement. Thermal cameras provide temperature measurement within an area at a high spatial resolution, which makes up for the shortcomings of other temperature measurement tools that only provide point measurements. Thermal imaging allows canopy temperature measurement under changing environmental conditions. It also allows fast measurement of a large number of plots at the field scale. Thermal cameras can be used to study patterns of temperature variations within the canopy even if the temperature measurement is not well calibrated. On the other hand, because thermal imaging is strongly affected by the surrounding objects and the environment, accurate calibration for temperature measurement can be extremely difficult. In many cases, corrections based on soil background and weather conditions, such as transient wind and cloud cover, need to be considered in calibration.
Spectral imaging
Spectral imaging is very promising for plant high-throughput phenotypic analysis. In recent years, spectral imaging technologies have been continuously improved, providing higher spectral and spatial resolutions and stronger stability under varying environmental conditions. This technology has been widely used in controlled environments. When carried on aerial platforms (helicopters, balloons, drones, and cranes), it also has been proven effective in large-scale field phenotyping. A factor limiting wide adoption of the spectroscopic imaging technology in high-throughput phenotyping is the high cost of the equipment and the need for proper illumination, which further increases the cost. Perhaps the most important limiting factor is the huge amount of data it generates, which requires large computing power and storage capabilities. The situation is worsened when images are taken from multiple cameras and when motions of the camera are involved. To reduce the complexity, appropriate wavelength selection and data dimensional reduction schemes must be studied. To a larger extend, this is a “big data” problem that requires a tremendous amount of work in the future.
3D imaging technique
For both controlled environments and field conditions, 3D imaging technology is a powerful tool for the measurement of complex morphological features, such as branch structure, leaf angle distribution, and root structure. However, the scale, resolution, throughput, accessibility, and cost of this technology often limit its use
[8].
Stereovision is highly dependent on the algorithm. Performance of stereovision in plant phenotyping is often affected by the lighting condition and optical features of the plant, such as texture of plant surfaces. Some stereovision systems require rotations of the plant, which requires a control system. When a stereo system is deployed to field, canopy movement due to wind becomes a major problem.
For root phenotyping, tomographic technologies have shown advantages in the detection and reconstruction of root systems, measurement of water transportation, and quantitative analysis of photosynthesis in plants. However, MRI is very sensitive to the type of media used for plant growth. Furthermore, due to the high cost, low resolution, and long scanning time, these technologies cannot be applied on a large scale in the field.
Laser scanning requires relatively long imaging times. Use of ground-penetrating radar is generally limited to thick roots (at least 0.5 cm in diameter) at relatively shallow depths, depending on the soil type and is most effective on dry, sandy soils. These limitations make ground-penetrating radar useful only for measuring root biomass of woody species. X-ray CT applications in plant research have been strictly limited.
3D imaging technologies have not found wide applications mainly due to their high cost and complexity of data analytics—a “big data” problem. Again, with the advances in technology and “big data” research, a rapid development in plant phenotyping using 3D technologies can be expected.
Image processing techniques for plant phenotyping
To analyze the phenotypic characteristics of plants, we must first analyze and process the images acquired by various imaging technologies. Commonly used image processing procedures include image coding, transformation, compression, enhancement, restoration, segmentation, description and classification. In general, image segmentation is the most important preprocessing operation, followed by feature extraction. The target plant traits are then obtained from these features through data analysis.
Image segmentation
Image segmentation determines the quality of the final image analysis result. Therefore, it is one of the most important steps and, in many cases, the most challenging task in image processing
[78]. Thresholding is one of the oldest and most common techniques for segmentation. The thresholding methods group pixels by similarity, assuming that individual objects within an image contain pixels with similar properties. These thresholds can be empirically determined from training examples (supervised approach). They can also be computed using only the information in the image (unsupervised approach). Usually, an image is divided into sections with a threshold determined for each section. A histogram is often constructed based on pixel properties. Various statistical methods can then be used to derive the thresholds that separate the sections.
Edge detection is a common method to identify discontinuous areas. This segmentation technique relies on the assumption that the boundaries of objects of interest in the image correspond to sharp changes in pixel properties. Many edge detection methods use kernel-based filters which rely on calculated discrete derivatives.
Feature extraction
Feature extraction is a concept in computer vision that extracts the higher level of feature information from images to be used in the next step of machine learning. Common types of measurements extracted from imagery include those related to color (spectral), shape, and texture.
Color characteristics are the most common type of feature extracted from images, because color features are often relevant to specific objects or scenes. In addition, color characteristics are more robust than other visual features. As such, a more compact representation of the color in an image or a region of interest is often of great value. In the image realm, colors are represented using a particular color model. Commonly used color models include the RGB, hue-saturation-intensity, and hue-saturation-value models. The most common representation for spectral characteristics is that of the histogram that measures the global distribution of colors in these models.
Texture features refer to the parameters that quantify the spatial patterns of gray-scale or color changes within an image
[79]. The most commonly used texture classes are: (1) statistical, (2) transformational, and (3) model-based. In plant phenotyping, texture features are often used to identify various types of objects, such as leaves, stems, flowers, seeds, and fruits.
Data analytics
Data analytics is the central task in accomplishing specific plant phenotyping goals. For example, spectral data can be converted to different forms—difference spectra, ratio spectra, and derivative spectra—to identify healthy and diseased plants
[80]. Based on the understanding of reflectance characteristics, spectral signatures have been developed to correlate biochemical and biophysical plant parameters with plant health and vigor
[63]. Spectral vegetation indices (SVIs) have been developed and widely applied for monitoring, analyzing, and mapping temporal and spatial variations in vegetation. By calculating ratios between spectral bands, SVIs result in a reduction of data dimension for disease discrimination. As pigment concentration provides information on the physiological state of leaves, pigment-specific SVIs are likely to be useful in the detection of stresses caused by fungal diseases
[81].
Classification is a procedure to assign image or spectral signatures to characteristic groups or classes, and discriminate these groups from each other. Principal components analysis (PCA), spectral angle mapper (SAM) and machine learning methods, such as ANN or support vector machines, are the most common methods used for classification
[82]. Since disease epidemics and symptom development cause temporal and spatial changes in crop reflectance, most classification techniques developed for remote sensing can be used to detect disease-induced spectral changes. Rumpf et al.
[82] successfully differentiated foliar diseases of sugar beet in a pre-symptomatic stage using Support Vector Machines (SVMs). The SAM method compares vectors representing experimentally determined spectra with reference spectra, and calculates an angle to represent the degree of difference between the two at each pixel. Moshou et al.
[83] used ANNs to classify healthy from diseased wheat plants. Data mining techniques have been shown to be superior for distinguishing types of stresses and different diseases
[83]. Quin et al.
[84] differentiated healthy and canker-disease damaged citrus fruits with a spectral-based algorithm.
The most widely used approach to process hyperspectral images is the multivariate methods, such as partial least squares (PLS) that builds statistical models to map the hyperspectral data with the traits of interest. The PLS-based models often outperforms other models as both the spectral (phenotypic) and trait (genotypic) information are built into the models for calibration. However, for such multivariate methods, care must be taken to use a sufficiently large set of both phenotypic and genotypic data to build the model; otherwise poor and unreliable predictions may be obtained
[13]. Algorithms for developing the models may include 2D correlation plots, partial least squares regression, PCA, support vector machines, neural networks and other machine learning approaches. Once calibration models are successfully developed and validated, they can be employed in routine analyses to predict phenotypic values of external data sets, and further used in combination with environmental and genotypic data to make breeding decisions
[85].
In thermography, absolute temperature, temperature differences between leaf tissue and air temperature, and maximal temperature differences of plant tissue are effective parameters
[86]. Visualization of isotherms is also useful for accentuation of hot and cold spots due to infections by leaf pathogens. Approaches developed in other fields, such as medical science, might be adaptable to plant phenotyping. For example, automated detection of temperature anomalies that indicate zones of diseased tissue may have a great potential for early screening of diseases in plants.
Summary and future trends
This article provides an overview of various imaging technologies applied in high-throughput plant phenotyping. Plant phenotyping tasks are accomplished on three types of platforms: stationary, ground-based, and aerial and traits measured include morphological, physiological and pathological traits.
Visible light imaging is the most commonly used imaging technology in plant phenotyping. It has been used in shoot and root phenotyping at different scales, from individual leaves/roots to whole canopy and root systems. Visible light imaging provides direct measures of plant morphological traits, and indirect measures of physiological and pathological traits. Both fluorescence and thermal imaging technologies can be used to measure plant physiological and pathological traits. However, thermal imaging only detects the traits involving plant canopy temperature changes, whereas fluorescence imaging detects traits closely related to plant’s photosynthetic function and its associated metabolism. Thus, fluorescence and thermal imaging are useful tools to detect different types of physiological deficiencies and pathological damage to plants. Combined use of these technologies can provide more complete evaluation of the plant health status.
Multispectral and hyperspectral imaging systems simultaneously acquire spatial and spectral information in the visible to near-infrared wave bands. Although plant information directly provided by the spectral imaging systems are morphological in nature, multiple physiological and pathological traits may be inferred owing to the wide wavelength range. Although stereovision and other 3D imaging technologies are costly to use and require sophisticated image processing techniques, they are promising tools for three-dimensional reconstructions of plant shoot and root systems, which will greatly enhance crop modeling and comprehensive studies for plant improvement.
While each imaging technology has its advantages and limitations, common image processing analytics can be developed. As all types of imaging technologies are increasingly adapted for plant phenotyping, “big data” analytics has become a critical issue that requires intensive research in the near future.
Finally, successful plant phenotyping can be achieved only by integrating the expertise of a multidisciplinary team of plant biologists, physicists, mathematicians, and engineers. Platforms of different scales must be made available to researchers. Experiences obtained from fields and growth facilities must be integrated. Effective and frequent exchanges of research results will help to quickly eliminate the bottleneck of phenotyping to match the rapid development in genomic research.
The Author(s) 2018. Published by Higher Education Press. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0)
PDF
(234KB)
