Application of light detection and ranging and ultrasonic sensors to high-throughput phenotyping and precision horticulture: current status and challenges

Abstract

Ultrasonic and light detection and ranging (LiDAR) sensors have been some of the most deeply investigated sensing technologies within the scope of digital horticulture. They can accurately estimate geometrical and structural parameters of the tree canopies providing input information for high-throughput phenotyping and precision horticulture. A review was conducted in order to describe how these technologies evolved and identify the main investigated topics, applications, and key points for future investigations in horticulture science. Most research efforts have been focused on the development of data acquisition systems, data processing, and high-resolution 3D modeling to derive structural tree parameters such as canopy volume and leaf area. Reported applications of such sensors for precision horticulture were restricted to real-time variable-rate solutions where ultrasonic or LiDAR sensors were tested to adjust plant protection product or fertilizer dose rates according to the tree volume variability. More studies exploring other applications in site-specific management are encouraged; some that integrates canopy sensing data with other sources of information collected at the within-grove scale (e.g., digital elevation models, soil type maps, historical yield maps, etc.). Highly accurate 3D tree models derived from LiDAR scanning demonstrate their great potential for tree phenotyping. However, the technology has not been widely adopted by researchers to evaluate the performance of new plant varieties or the outcomes from different management practices. Commercial solutions for tree scanning of whole groves, orchards, and nurseries would promote such adoption and facilitate more applied research in plant phenotyping and precision horticulture.

References

[1] Berk, P., Hocevar, M., Stajnko, D. & Belsak, A. Development of alternative plant protection product application techniques in orchards, based on measurement sensing systems: a review. Comput. Electron. Agric. [Internet] 124, 273-288 (2016).

[2] Dworak, V., Selbeck, J. & Ehlert, D. Ranging sensors for vehicle-based measurement of crop stand and orchard parameters: a review. Trans. ASABE 54, 1497-1510 (2011).

[3] Gil, E. et al. Advanced technologies for the improvement of spray application techniques in spanish viticulture: an overview. Sensors [Internet] 14, 691-708 (2014).

[4] Lin, Y. LiDAR: an important tool for next-generation phenotyping technology of high potential for plant phenomics?. Comput. Electron. Agric. 119, 61-73 (2015).

[5] Rosell-Polo, J. R., & Sanz, R. A review of methods and applications of the geometric characterization of tree crops in agricultural activities. Comput. Electron. Agric. 81, 124-141 (2012).

[6] Lee, W. S. et al. Sensing technologies for precision specialty crop production. Comput. Electron. Agric. [Internet] 74, 2-33 (2010).

[7] Vázquez-Arellano, M., Griepentrog, H., Reiser, D. & Paraforos, D. 3-D imaging systems for agricultural applications-a review. Sensors [Internet] 16, 618 (2016).

[8] Sutton, T. B. & Unrath, C. R. Evaluation of the Tree-Row-Volume Model for full-season pesticide application on apples. Plant Dis. 72, 629-632 (1988).

[9] Sutton, T. B. & Unrath, C. R. Evaluation of the Tree-Row-Volume concept with density adjuvants in relation to spray deposits in apple orchards. Plant Dis. 68, 480-484 (1984).

[10] Byers, R. E. Tree-row-volume spraying rate calculator for apples. HortScience 22, 506-507 (1987).

[11] Byers, R. E. et al. Effect of apple tree size and canopy density on spray chemical deposit. HortScience 19, 93-94 (1984).

[12] McConnell, R. L., Elliot, K. C., Blizzard, S. H. & Koster, K. H. Electronic measurement of tree-row-volume. Agric. Electron. 1, 85-90 (1983).

[13] Giles, D. K., Delwiche, M. J. & Dodd, R. B. Sprayer control by sensing orchard crop characteristics: orchard architecture and spray liquid savings. J. Agric. Eng. Res. 43(C), 271-289 (1989).

[14] Giles, D. K., Delwiche, M. J. & Dodd, R. B. Electronic measurement of tree canopy volume. Trans. ASAEE 31, 264-273 (1988).

[15] Giles, D. K., Delwiche, M. J. & Dodd, R. B. Control of orchard spraying based on electronic sensing of target characteristics. Trans. ASABE 30, 1624-1630 (1987).

[16] Moltó, E., Martín, B. & Gutiérrez, A. Pesticide loss reduction by automatic adaptation of spraying on globular trees. J. Agric. Eng. Res. [Internet] 78, 35-41 (2001).

[17] Moltó, E., Martín, B. & Gutiérrez, A. Design and testing of an automatic machine for spraying at a constant distance from the tree canopy. J. Agric. Eng. Res. [Internet] 77, 379-384 (2000).

[18] Balsari, P., Marucco, P. & Tamagnone, M. A crop identification system (CIS) to optimise pesticide applications in orchards. J. Hortic. Sci. Biotechnol. [Internet] 84, 113-116 (2009).

[19] Escolà, A. et al. Performance of an ultrasonic ranging sensor in apple tree canopies. Sensors 11, 2459-2477 (2011).

[20] Osterman, A., Godeša, T., Hočevar, M., Širok, B. & Stopar, M. Real-time positioning algorithm for variable-geometry air-assisted orchard sprayer. Comput. Electron. Agric. [Internet] 98, 175-182 (2013).

[21] Wangler, R. J., McConnell, R. E., Fowler, K. L. in Optics in Agriculture and Forestry Society of Photo-optical Instrumentation Engineers-SPIE [Internet] (eds DeShazer, J. A. & Meyer, G. E.) 261-272 (1993). http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=1004593

[22] Wangler, R. J., Fowler, K. L. & McConnell, R. E. Object sensor and method for use in controlling an agricultural sprayer. US patent 5,278,423, 0-4 (1994).

[23] Escolà, A. et al. Variable rate sprayer. Part 1-orchard prototype: design, implementation and validation. Comput. Electron. Agric. [Internet] 95, 122-135 (2013).

[24] Chen, Y., Zhu, H. & Ozkan, H. E. Development of a variable-rate sprayer with laser scanning sensor to synchronize spray outputs to tree structures. Trans. ASABE 55, 773-781 (2012).

[25] Walklate, P. J., Richardson, G. M., Baker, D. E., Richards, P. A. & Cross, J. V. in Advances in Laser Remote Sensing for Terrestrial and Oceanographic Applications [Internet] (eds Narayanan, R. M. & Kalshoven, Jr. J. E.) 143-151 (1997) [cited 2013 May 23]. http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=924985

[26] Walklate, P. J., Cross, J. V., Richardson, G. M., Murray, R. A. & Baker, D. E. Comparison of different spray volume deposition models using LIDAR measurements of apple orchards. Biosyst. Eng. [Internet] 82, 253-267 (2002).

[27] Cross, J. V., Walklate, P. J., Murray, R. A., & Richardson, G. M. Spray deposits and losses in different sized apple trees from an axial fan orchard sprayer: 2. Effects of spray quality. Crop Prot. 20, 333-343 (2001).

[28] Cross, J. V., Walklate, P. J., Murray, R. A. & Richardson, G. M. Spray deposits and losses in different sized apple trees from an axial fan orchard sprayer: 1. Effects of air volumetric flow rate. Crop Prot. 20, 13-30 (2001).

[29] Cross, J. V., Walklate, P. J., Murray, R. A. & Richardson, G. M. Spray deposits and losses in different sized apple trees from an axial fan orchard sprayer: 3. Effects of spray quality. Crop Prot. 22, 381-394 (2003).

[30] Cross, J. V. & Walklate, P. J. The UK PACE Scheme for adjusting the dose to suit apple crops. Agric. Eng. Int. CIGR EJournal. 10, 1-10 (2008).

[31] Walklate, P. J., Cross, J. V., & Pergher, G. Support system for efficient dosage of orchard and vineyard spraying products. Comput. Electron. Agric. 75, 355-362 (2011).

[32] Walklate, P. J., Cross, J. V., Richardson, G. M. & Baker, D. E. Optimising the adjustment of label-recommended dose rate for orchard spraying. Crop. Prot. 25, 1080-1086 (2006).

[33] Walklate, P. J., & Cross, J. V. Regulated dose adjustment of commercial orchard spraying products. Crop Prot. [Internet] 54, 65-73 (2013).

[34] Walklate, P. J., & Cross, J. V. An examination of Leaf-Wall-Area dose expression. Crop. Prot. 35, 132-134 (2012).

[35] Whitney, J. D., Miller, W. M., Wheaton, T. A., Salyani, M. & Schueller, J. K. Precision farming applications in Florida citrus. Appl. Eng. Agric. 15, 399-403 (1999).

[36] Schumann, A. W. & Zaman, Q. U. Software development for real-time ultrasonic mapping of tree canopy size. Comput. Electron. Agric. 47, 25-40 (2005).

[37] Zaman, Q. U. & Salyani, M. Effects of foliage density and ground speed on ultrasonic measurement of citrus tree volume. Appl. Eng. Agric. 20, 173-178 (2004).

[38] Zaman, Q. U. & Schumann, A. W. Performance of an ultrasonic tree volume measurement system in commercial citrus groves. Precis. Agric. 6, 467-480 (2005).

[39] Zaman, Q. U., Schumann, A. W. & Hostler, K. H. Estimation of citrus fruit yield using ultrasonically-sensed tree size. Appl. Eng. Agric. 22, 39-44 (2006).

[40] Schumann, A. W., Hostler, K. H., Buchanon, S. & Zaman, Q. U. Relating citrus canopy size and yield to precision fertilization. Proc. Fla. State Hortic. Soc. 119, 148-154 (2006).

[41] Schumann, A. W. et al. Variable rate granular fertilization of citrus groves: spreader performance with single-tree prescription zones. Appl. Eng. Agric. 22, 19-24 (2006).

[42] Zaman, Q. U., Schumann, A. W. & Miller, W. M. Variable rate nitrogen application in Florida citrus based on ultrasonically-sensed tree size. Appl. Eng. Agric. 21, 331-336 (2005).

[43] Tumbo, S. D., Salyani, M., Whitney, J. D., Wheaton, T. A. & Miller, W. M. Investigation of laser and ultrasonic ranging sensors for measurements of citrus canopy volume. Appl. Eng. Agric. 18, 367-372 (2002).

[44] Wei, J. & Salyani, M. Development of a laser scanner for measuring tree canopy characteristics: phase 1. Prototype Dev. Trans. ASABE 47, 2101-2108 (2004).

[45] Wei, J. & Salyani, M. Development of a laser scanner for measuring tree canopy characteristics: phase 2. Foliage density measurement. Trans. ASABE 48, 1595-1602 (2005).

[46] Lee, K. H. & Ehsani, R. Comparison of two 2D laser scanners for sensing object distances, shapes, and surface patterns. Comput. Electron. Agric. 60, 250-262 (2008).

[47] Lee, K. H. & Ehsani, R. A laser scanner based measurement system for quantification of citrus tree geometric characteristics. Appl. Eng. Agric. 25, 777-788 (2009).

[48] Solanelles, F. et al. An electronic control system for pesticide application proportional to the canopy width of tree crops. Biosyst. Eng. 95, 473-481 (2006).

[49] Gil, E., Escolà, A., Rosell-Polo, J. R., Planas, S. & Val, L. Variable rate application of plant protection products in vineyard using ultrasonic sensors. Crop Prot. 26, 1287-1297 (2007).

[50] Gil, E. et al. Variable rate sprayer. Part 2-vineyard prototype: design, implementation, and validation. Comput. Electron. Agric. 95, 136-150 (2013).

[51] Llorens, J., Gil, E., Llop, J., & Escolà, A. Variable rate dosing in precision viticulture: use of electronic devices to improve application efficiency. Crop Prot. 29, 239-248 (2010).

[52] Llorens, J., Gil, E., Llop, J. & Escolà, A. Ultrasonic and LIDAR sensors for electronic canopy characterization in vineyards: advances to improve pesticide application methods. Sensors 11, 2177-2194 (2011).

[53] Rosell-Polo, J. R. et al. Obtaining the three-dimensional structure of tree orchards from remote 2D terrestrial LIDAR scanning. Agric. Meteorol. 149, 1505-1515 (2009).

[54] Keightley, K. E., & Bawden, G. W. 3D volumetric modeling of grapevine biomass using Tripod LiDAR. Comput. Electron. Agric. 74, 305-312 (2010).

[55] Sanz, R., Llorens, J., Rosell-Polo, J. R., Gregorio, E. & Palacín, J. Characterisation of the LMS200 laser beam under the influence of blockage surfaces. Influence on 3D scanning of tree orchards. Sensors 11, 2751-2772 (2011).

[56] Bietresato, M., Carabin, G., Vidoni, R., Gasparetto, A., & Mazzetto, F. Evaluation of a LiDAR-based 3D-stereoscopic vision system for crop-monitoring applications. Comput. Electron. Agric. [Internet] 124, 1-13 (2016).

[57] Escolà, A. et al. Mobile terrestrial laser scanner applications in precision fruticulture/horticulture and tools to extract information from canopy point clouds. Precis. Agric. [Internet] 18, 111-132 (2017).

[58] Colaço, A. F., Trevisan, R. G., Molin, J. P., Rosell-Polo, J. R. & Escolà, A. A method to obtain orange crop geometry information using a mobile terrestrial laser scanner and 3D modeling. Remote Sens. [Internet] 9, 763 (2017).

[59] Underwood, J. P., Hung, C., Whelan, B. M. & Sukkarieh, S. Mapping almond orchard canopy volume, flowers, fruit and yield using LiDAR and vision sensors. Comput. Electron. Agric. 130, 83-96 (2016).

[60] Moorthy, I. et al. Field characterization of olive (Olea europaea L.) tree crown architecture using terrestrial laser scanning data. Agric. Meteorol. 151, 204-214 (2011).

[61] Méndez, V., Catalán, H., Rosell-Polo, J. R., Arnó, J. & Sanz, R. LiDAR simulation in modelled orchards to optimise the use of terrestrial laser scanners and derived vegetative measures. Biosyst. Eng. 115, 7-19 (2013).

[62] Méndez, V. et al. SIMLIDAR-Simulation of LIDAR performance in artificially simulated orchards. Biosyst. Eng. 111, 72-82 (2012).

[63] Méndez, V., Rosell-Polo, J. R., Sanz, R., Escolà, A. & Catalán, H. Deciduous tree reconstruction algorithm based on cylinder fitting from mobile terrestrial laser scanned point clouds. Biosyst. Eng. 124, 78-88 (2014).

[64] Wellington, C., Campoy, J., Khot, L. R. & Ehsani, R. Orchard tree modeling for advanced sprayer control and automatic tree inventory. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Agricultural Robotics (2012). IEEE, Vilamoura, Portugal.

[65] Bargoti, S., Underwood, J. P., Nieto, J. I. & Sukkarieh, S. A pipeline for trunk detection in trellis structured apple orchards. J. F. Robot 32, 1075-1094 (2015).

[66] Stein, M., Bargoti, S. & Underwood, J. P. Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors 16, 1915 (2016).

[67] Méndez, V., Rosell-Polo, J. R., Pascual, M. & Escolà, A. Multi-tree woody structure reconstruction from mobile terrestrial laser scanner point clouds based on a dual neighbourhood connectivity graph algorithm. Biosyst. Eng. 148, 34-47 (2016).

[68] Yandún., F., Salvo del Pedregal, J., Prieto, P. A., Torres-Torriti, M. & Auat Cheein, F. A. LiDAR and thermal images fusion for ground-based 3D characterisation of fruit trees. Biosyst. Eng. 151, 479-494 (2016).

[69] Hosoi, F. & Omasa, K. Voxel-based 3-D modeling of individual trees for estimating leaf area density using. IEEE Trans. Geosci. Remote Sens. 44, 3610-3618 (2006).

[70] Auat Cheein, F. A. & Guivant, J. SLAM-based incremental convex hull processing approach for treetop volume estimation. Comput. Electron. Agric. 102, 19-30 (2014).

[71] Rosell-Polo, J. R. et al. A tractor-mounted scanning LIDAR for the non-destructive measurement of vegetative volume and surface area of tree-row plantations: a comparison with conventional destructive measurements. Biosyst. Eng. 102, 128-134 (2009).

[72] Auat Cheein, F. A. et al. Real-time approaches for characterization of fully and partially scanned canopies in groves. Comput. Electron. Agric. 118, 361-371 (2015).

[73] Sanz, R., Rosell-Polo, J. R., Llorens, J., Gil, E. & Planas, S. Relationship between tree row LIDAR-volume and leaf area density for fruit orchards and vineyards obtained with a LIDAR 3D dynamic measurement system. Agric. Meteorol. 171-172, 153-162 (2013).

[74] Sanz, R. et al. Innovative LIDAR 3D dynamic measurement system to estimate fruit-tree leaf area. Sensors 11, 5769-5791 (2011).

[75] Arnó, J. et al. Leaf area index estimation in vineyards using a ground-based LiDAR scanner. Precis. Agric. 14, 290-306 (2013).

[76] Pforte, F., Selbeck, J. & Hensel, O. Comparison of two different measurement techniques for automated determination of plum tree canopy cover. Biosyst. Eng. 113, 325-333 (2012).

[77] Arnó, J., Escolà, A., Masip, J. & Rosell-Polo, J. R. Influence of the scanned side of the row in terrestrial laser sensor applications in vineyards: practical consequences. Precis. Agric. 16, 119-128 (2015).

[78] Arnó, J., Escolà, A. & Rosell-Polo, J. R. Setting the optimal length to be scanned in rows of vines by using mobile terrestrial laser scanners. Precis. Agric. [Internet] 18, 145-151 (2017).

[79] Del-Moral-Martínez, I. et al. Mapping vineyard leaf area using mobile terrestrial laser scanners: should rows be scanned on-the-go or discontinuously sampled?. Sensors 16, 119 (2016).

[80] Pallejà, T. et al. Sensitivity of tree volume measurement to trajectory errors from a terrestrial LIDAR scanner. Agric. Meteorol. 150, 1420-1427 (2010).

[81] Del-Moral-Martínez, I. et al. Georeferenced scanning system to estimate the leaf wall area in tree crops. Sensors 15, 8382-8405 (2015).

[82] Underwood, J. P., Jagbrant, G., Nieto, J. I. & Sukkarieh, S. Lidar-based tree recognition and platform localization in orchards. J. F. Robot. 32, 1056-1074 (2015).

[83] Underwood, J. P., Hill, A., Peynot, T. & Scheding, S. J. Error modeling and calibration of exteroceptive sensors for accurate mapping applications. J. F. Robot. 27, 2-20 (2010).

[84] Díaz-Varela, R., de la Rosa, R., León, L. & Zarco-Tejada, P. High-resolution airborne UAV imagery to assess olive tree crown parameters using 3D photo reconstruction: application in breeding trials. Remote Sens. 7, 4213-4232 (2015).

[85] Torres-Sánchez, J., López-Granados, F., Serrano, N., Arquero, O. & Peña, J. M. High-throughput 3-D monitoring of agricultural-tree plantations with unmanned aerial vehicle (UAV) technology. Hassan QK, editor. PLoS ONE 10, e0130479 (2015).

[86] Rosell-Polo, J. R. et al. Advances in structured light sensors applications in precision agriculture and livestock farming. Adv. Agron. 133, 71-112 (2015).

[87] Rosell-Polo, J. R. et al. Kinect v2 sensor-based mobile terrestrial laser scanner for agricultural outdoor applications. IEEE/ASME Trans. Mechatron. 4435(c), 1- 1 (2017).

[88] Mann, K. K., Schumann, A. W. & Obreza, T. A. Delineating productivity zones in a citrus grove using citrus production, tree growth and temporally stable soil data. Precis. Agric. 12, 457-472 (2011).

[89] Velázquez-Martí, B., Estornell, J., López-Cortés, I. & Martí-Gavilá, J. Calculation of biomass volume of citrus trees from an adapted dendrometry. Biosyst. Eng. 112, 285-292 (2012).

[90] Miranda-Fuentes, A., Llorens, J., Gamarra-Diezma, J. L., Gil-Ribes, J. A. & Gil, E. Towards an optimized method of olive tree crown volume measurement. Sensors 15, 3672-3687 (2015).

[91] Li, L., Zhang, Q. & Huang, D. A review of imaging techniques for plant phenotyping. Sensors 14, 20078-20111 (2014).

[92] Bai, G., Ge, Y., Hussain, W., Baenziger, P. S. & Graef, G. A multi-sensor system for high throughput field phenotyping in soybean and wheat breeding. Comput. Electron. Agric. 128, 181-192 (2016).

[93] Maghsoudi, H., Minaei, S., Ghobadian, B. & Masoudi, H. Ultrasonic sensing of pistachio canopy for low-volume precision spraying. Comput. Electron. Agric. 112, 149-160 (2015).