Phenotypic dynamics and temporal heritability of tomato architectural traits using an unmanned ground vehicle-based plant phenotyping system

Abstract

Large-scale manual measurements of plant architectural traits in tomato growth are laborious and subjective, hindering deeper understanding of temporal variations in gene expression heterogeneity. This study develops a high-throughput approach for characterizing tomato architectural traits at different growth stages and mapping temporal broad-sense heritability using an unmanned ground vehicle-based plant phenotyping system. The SegFormer with fusion of multispectral and depth imaging modalities was employed to semantically segment plant organs from the registered RGB-D and multispectral images. Organ point clouds were then generated and clustered into instances. Finally, six key architectural traits, including fruit spacing (FS), inflorescence height (IH), stem thickness (ST), leaf spacing (LS), total leaf area (TLA), and leaf inclination angle (LIA) were extracted and the temporal broad-sense heritability folds were plotted. The root mean square errors (RMSEs) of the estimated FS, IH, ST, and LS were 0.014, 0.043, 0.003, and 0.015 m, respectively. The visualizations of the estimated TLA and LIA matched the actual growth trends. The broad-sense heritability of the extracted traits exhibited different trends across the growth stages: (i) ST, IH, and FS had a gradually increased broad-sense heritability over time, (ii) LS and LIA had a decreasing trend, and (iii) TLA showed fluctuations (i.e. an M-shaped pattern) of the broad-sense heritability throughout the growth period. The developed system and analytical approach are promising tools for accurate and rapid characterization of spatiotemporal changes of tomato plant architecture in controlled environments, laying the foundation for efficient crop breeding and precision production management in the future.

Acknowledgements

This work was funded by the Science and Technology Project of the Ministry of Agriculture and Rural Affairs, the Fundamental Research Funds for the Central Universities (226-2022-00217) and the National Natural Science Foundation of China (32371985).

Author contributions

P.X., X.Y., and L.F. designed the study, conducted the experiment, and wrote the manuscript. T.W. and J.Z. provided agronomic guidance during the experiments and manuscript writing process. Y.J. reviewed and revised the manuscript. H.C. supervised experiments at all stages and performed revisions of the manuscript. All authors read and approved the final manuscript.

Data availability

The pipeline code and example data related to this project are available as open source on GitHub (https://github.com/DigBigPigForU/Tomato-architectural-trait-extraction).

Conflict of interest statement

The authors declared no conflict of interest.

Supplementary data

Supplementary data is available at Horticulture Research online.

References

[1] Reynolds M, Atkin OK, Bennett M. et al. Feature review address-ing research bottlenecks to crop productivity. Trends Plant Sci. 2021; 26:607-30

[2] Guo W, Chen L, Herrera-Estrella L. et al. Altering plant archi-tecture to improve performance and resistance. Trends Plant Sci. 2020; 25:1154-70

[3] Laurans M, Munoz F, Charles-Dominique T. et al. Why incorpo-rate plant architecture into trait-based ecology? 2024; 4239:524-36

[4] Perez RPA, Fournier C, Cabrera-Bosquet L. et al. Changes in the vertical distribution of leaf area enhanced light interception effi-ciency in maize over generations of selection. Plant Cell Environ. 2019; 42:2105-19

[5] Zhang Y, Henke M, Li Y. et al. Analyzing the impact of greenhouse planting strategy and plant architecture on tomato plant physi-ology and estimated dry matter. Front Plant Sci. 2022b; 13:828252

[6] Shah DU, Reynolds TPS, Ramage MH. The strength of plants: theory and experimental methods to measure the mechanical properties of stems. JExp Bot. 2017; 68:4497-516

[7] Dong N-Q, Lin H-X. Compact plants enhance productivity. Nat Plants. 2022; 8:1335-6

[8] Lyu J. Edit compactness. Nat Plants. 2020; 6:180-0

[9] Li M, Klein LL, Duncan KE. et al. Characterizing 3D inflorescence architecture in grapevine using X-ray imaging and advanced morphometrics: implications for understanding cluster density. JExp Bot. 2019; 70:6261-76

[10] Perilleux C, Lobet G, Tocquin P. Inflorescence development in tomato: gene functions within a zigzag model. Front Plant Sci. 2014; 5:121

[11] Slafer GA, Foulkes MJ, Reynolds MP. et al. A ’wiring diagram’ for sink strength traits impacting wheat yield potential. JExp Bot. 2023; 74:40-71

[12] Cao Y, Zhong Z, Wang H. et al. Leaf angle: a target of genetic improvement in cereal crops tailored for high-density planting. Plant Biotechnol J. 2022; 20:426-36

[13] Niinemets U, Keenan TF, Hallik L. A worldwide analysis of within-canopy variations in leaf structural, chemical and physiological traits across plant functional types. New Phytol. 2015; 205:973-93

[14] Richards RA. Selectable traits to increase crop photosynthesis and yield of grain crops. JExp Bot. 2000; 51:447-58

[15] Sperry JS, Hacke UG, Oren R. et al. Water deficits and hydraulic limits to leaf water supply. Plant Cell Environ. 2002; 25:251-63

[16] Tardieu F, Tuberosa R. Dissection and modelling of abiotic stress tolerance in plants. Curr Opin Plant Biol. 2010; 13:206-12

[17] Ward SP, Leyser O. Shoot branching. Curr Opin Plant Biol. 2004; 7: 73-8

[18] Kuijken RCP, van Eeuwijk FA, Marcelis LFM. et al. Root pheno-typing: from component trait in the lab to breeding. JExp Bot. 2015; 66:5389-401

[19] Nyquist WE. Estimation of heritability and prediction of selec-tion response IN plant-populations. Crit Rev Plant Sci. 1991; 10: 235-322

[20] Basu U, Parida SK. Restructuring plant types for developing tailor-made crops. Plant Biotechnol J. 2023; 21:1106-22

[21] Huang X, Hilscher J, Stoger E. et al. Modification of cereal plant architecture by genome editing to improve yields. Plant Cell Rep. 2021; 40:953-78

[22] Purugganan MD, Jackson SA. Advancing crop genomics from lab to field. Nat Genet. 2021;53:595-+

[23] Varshney RK, Bohra A, Yu J. et al. Feature review designing future crops: genomics-assisted breeding comes of age. Trends Plant Sci. 2021; 26:631-49

[24] Bao Y, Tang L, Srinivasan S. et al. Field-based architectural traits characterisation of maize plant using time-of-flight 3D imaging. Biosyst Eng. 2019a; 178:86-101

[25] Ci JC, van Henten EJ, Wang X. et al.SSL-NBV:a self-supervised-learning-based next-best-view algorithm for efficient 3D plant reconstruction by a robot. Comput Electron Agric. 2025; 233:110121

[26] Wang X, Burusa AK, Rapado-Rincón D. et al. Multiple object tracking with multi-view active vision to effectively find plant nodes in a cluttered tomato greenhouse. Comput Electron Agric. 2025; 234:110266

[27] Hu KW, Pan YQ, Liu TH. et al. Phenobot: an autodigital modeling system for in situ phenotyping in horticulture. Adv Intell Syst. 2025; 12:2400665

[28] Begot L, Slavkovic F, Oger M. et al. Precision phenotyping of nectar-related traits using X-ray micro computed tomography. Cells. 2022; 11:3452

[29] Choi HB, Park JK, Park SH. et al. NeRF-based 3D reconstruction pipeline for acquisition and analysis of tomato crop morphology. Front Plant Sci. 2024; 15:1439086

[30] van der Niet T, Zollikofer CPE, de Leon MSP. et al. Three-dimensional geometric morphometrics for studying floral shape variation. Trends Plant Sci. 2010; 15:423-6

[31] Badgujar CM, Poulose A, Gan H. Agricultural object detection with you only look once (YOLO) algorithm: a bibliometric and systematic literature review. Comput Electron Agric. 2024; 223: 109090

[32] Xie K, Zhu JZ, Ren H. et al. Delving into the potential of deep learning algorithms for point cloud segmentation at organ level in plant phenotyping. Remote Sens. 2024a; 16:28

[33] Wang ZB, Cui J, Zhu Y. Review of plant leaf recognition. Artif Intell Rev. 2023; 56:4217-53

[34] Zhang JW, Li C, Rahaman MM. et al. A comprehensive review of image analysis methods for microorganism counting: from classical image processing to deep learning approaches. Artif Intell Rev. 2022a; 55:2875-944

[35] Islam MP, Hatou K. TheLR531v1-a deep learning multi-branch CNN architecture for day-night automatic segmentation of hor-ticultural crops. Comput Electron Agric. 2023; 204:107557

[36] Ma R, Fuentes A, Yoon S. et al. Local refinement mechanism for improved plant leaf segmentation in cluttered backgrounds. Front Plant Sci. 2023; 14:1211075

[37] Rong Q, Hu C, Hu X. et al. Picking point recognition for ripe tomatoes using semantic segmentation and morphological pro-cessing. Comput Electron Agric. 2023; 210:107923

[38] Shi W, van de Zedde R, Jiang H. et al. Plant-part segmentation using deep learning and multi-view vision. Biosyst Eng. 2019; 187: 81-95

[39] Shoaib M, Hussain T, Shah B. et al. Deep learning-based segmen-tation and classification of leaf images for detection of tomato plant disease. Front Plant Sci. 2022; 13:1031748

[40] Liu C, Feng Q, Sun Y. et al. YOLACTFusion: an instance seg-mentation method for RGB-NIR multimodal image fusion based on an attention mechanism. Comput Electron Agric. 2023; 213: 108186

[41] Xie P, Du R, Ma Z. et al. Generating 3D multispectral point clouds of plants with fusion of snapshot spectral and RGB-D images. Plant Phenomics. 2023; 5:0040

[42] Aygunes B, Cinbis RG, Aksoy S. Weakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classification. ISPRS J Photogramm Remote Sens. 2021; 176:262-74

[43] Wei Y, He Y, Li X. Tea moisture content detection with multispec-tral and depth images. Comput Electron Agric. 2021; 183:106082

[44] Xu P, Fang N, Liu N. et al. Visual recognition of cherry tomatoes in plant factory based on improved deep instance segmentation. Comput Electron Agric. 2022; 197:106991

[45] Cao J, Leng H, Lischinski D. et al. ShapeConv: shape-aware con-volutional layer for indoor RGB-D semantic segmentation. In: 18th IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada: IEEE, 2021,7068-77

[46] Chen L-Z, Lin Z, Wang Z. et al. Spatial information guided con-volution for real-time RGBD semantic segmentation. IEEE Trans Image Process. 2021; 30:2313-24

[47] Chen X, Lin K-Y, Wang J. et al. Bi-directional cross-modality feature propagation with separation-and-aggregation gate for RGB-D semantic segmentation. In: Vedaldi A, Bischof H, Brox T, Frahm JM,eds. Computer Vision - ECCV 2020. ECCV 2020. Lecture Notes in Computer Science, Vol 12356. Springer International Publishing: Cham, 2020,

[48] Deng F, Feng H, Liang M. et al. FEANet: feature-enhanced atten-tion network for RGB-thermal real-time semantic segmentation. In: IEEE/RSJ International Conference on Intelligent Robots and Sys-tems (IROS). Prague, Czech Republic: IEEE, 2021,4467-73

[49] Zhang Q, Zhao S, Luo Y. et al. ABMDRNet: adaptive-weighted bi-directional modality difference reduction network for RGB-T semantic segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA: IEEE Computer Society, 2021,2633-42

[50] Paulus S. Measuring crops in 3D: using geometry for plant phenotyping. Plant Methods. 2019; 15:13

[51] Saeed F, Sun SP, Rodriguez-Sanchez J. et al. Cotton plant part 3D segmentation and architectural trait extraction using point voxel convolutional neural networks. Plant Methods. 2023; 19:23

[52] Serouart M, Lopez-Lozano R, Daubige G. et al. Analyzing changes in maize leaves orientation due to GxExM using an automatic method from RGB images. Plant Phenomics. 2023; 5:0046

[53] Xiang L, Bao Y, Tang L. et al. Automated morphological traits extraction for sorghum plants via 3D point cloud data analysis. Comput Electron Agric. 2019; 162:951-61

[54] Xin BL, Smolenová K, Bartholomeus H. et al. An automatic 3D tomato plant stemwork phenotyping pipeline at internode level based on tree quantitative structural modelling algorithm. Comput Electron Agric. 2024; 227:109607

[55] Zhu TY, Ma XD, Guan HO. et al. A calculation method of phe-notypic traits based on three-dimensional reconstruction of tomato canopy. Comput Electron Agric. 2023a; 204:107515

[56] Zhu TY, Ma XD, Guan HO. et al. A method for detecting tomato canopies’ phenotypic traits based on improved skeleton extrac-tion algorithm. Comput Electron Agric. 2023b; 214:108285

[57] Zhu YH, Gu Q, Zhao YY. et al. Quantitative extraction and eval-uation of tomato fruit phenotypes based on image recognition. Front Plant Sci. 2022; 13:859290

[58] Atefi A, Ge Y, Pitla S. et al. Robotic technologies for high-throughput plant phenotyping: contemporary reviews and future perspectives. Front Plant Sci. 2021; 12:611940

[59] Bao Y, Zarecor S, Shah D. et al. Assessing plant performance in the Enviratron. Plant Methods. 2019b; 15:117-7

[60] Martin J, Ansuategi A, Maurtua I. et al. A generic ROS-based control architecture for pest inspection and treatment in green-houses using a mobile manipulator. IEEE Access. 2021; 9:94981-95

[61] Yamati FRI, Bömer J, Noack N. et al. Configuration of a multi-sensor platform for advanced plant phenotyping and disease detection: case study on Cercospora leaf spot in sugar beet. Smart Agric Technol. 2025; 10:18

[62] Cho S, Kim T, Jung D-H. et al. Plant growth information measure-ment based on object detection and image fusion using a smart farm robot. Comput Electron Agric. 2023; 207:107703

[63] Li J, Qin H, Wang J. et al. OpenStreetMap-based autonomous navigation for the four wheel-legged robot via 3D-Lidar and CCD camera. IEEE Trans Ind Electron. 2022; 69:2708-17

[64] Wang T, Chen B, Zhang Z. et al. Applications of machine vision in agricultural robot navigation: a review. Comput Electron Agric. 2022; 198:107085

[65] Zhang J, Liu H, Yang K. et al. CMX: cross-modal fusion for RGB-X semantic segmentation with transformers. IEEE Trans Intell Transp Syst. 2023; 24:14679-94

[66] Balakrishnan G, Zhao A, Sabuncu MR. et al. VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging. 2019; 38:1788-800

[67] Dosovitskiy A, Fischer P, Ilg E. et al. FlowNet: learning optical flow with convolutional networks. In: IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015,2758-66

[68] Cardellicchio A, Solimani F, Dimauro G. et al. Detection of tomato plant phenotyping traits using YOLOv5-based single stage detectors. Comput Electron Agric. 2023; 207:107757

[69] Yao JJ, Gong Y, Xia ZY. et al. Facility of tomato plant organ segmentation and phenotypic trait extraction via deep learning. Comput Electron Agric. 2025; 231:109957

[70] Rossi R, Costafreda-Aumedes S, Leolini L. et al. Implementation of an algorithm for automated phenotyping through plant 3D-modeling: a practical application on the early detection of water stress. Comput Electron Agric. 2022; 197:106937

[71] Wang Y, Li J. Molecular basis of plant architecture. Annu Rev Plant Biol. 2008; 59:253-79

[72] Mantilla-Perez MB, Fernandez MGS. Differential manipulation of leaf angle throughout the canopy: current status and prospects. JExp Bot. 2017; 68:5699-717

[73] Zhao D, Luan Y, Shi W. et al. Melatonin enhances stem strength by increasing lignin content and secondary cell wall thickness in herbaceous peony. JExp Bot. 2022; 73:5974-91

[74] Teichmann T, Muhr M. Shaping plant architecture. Front Plant Sci. 2015; 6:233

[75] Wang YH, Li JY. Genes controlling plant architecture. Curr Opin Biotechnol. 2006; 17:123-9

[76] Wang B, Smith SM, Li JY. Genetic regulation of shoot architec-ture. In: Merchant SS,ed. Annual Review of Plant Biology, Vol 69. Annual Reviews: Palo Alto, 2018,437-68

[77] Anest A, Charles-Dominique T, Maurin O. et al. Evolving the structure: climatic and developmental constraints on the evolu-tion of plant architecture. A case study in Euphorbia. New Phytol. 2021; 231:1278-95

[78] van der Meer M, Lee H, de Visser PHB. et al. Consequences of interplant trait variation for canopy light absorption and photosynthesis. Front Plant Sci. 2023; 14:1012718

[79] Chitwood DH, Kumar R, Ranjan A. et al. Light-induced indetermi-nacy alters shade-avoiding tomato leaf morphology. Plant Physiol. 2015; 169:2030-47

[80] Chen T-W, Nguyet TM, N., Kahlen K., & Stuetzel H. Quantifica-tion of the effects of architectural traits on dry mass production and light interception of tomato canopy under different tem-perature regimes using a dynamic functional-structural plant model. JExp Bot. 2014; 65:6399-410

[81] Xie P, Ma Z, Du R. et al. An unmanned ground vehicle phenotyping-based method to generate three-dimensional mul-tispectral point clouds for deciphering spatial heterogeneity in plant traits. Mol Plant. 2024b; 17:1624-38

[82] Guan RP, Ristic B, Wang L. et al. KLD sampling with Gmapping proposal for Monte Carlo localization of mobile robots. Inf Fusion. 2019; 49:79-88

[83] Hsu D, Latombe J-C, Kurniawati H. On the probabilistic founda-tions of probabilistic roadmap planning. IntJRobotRes. 2006; 25: 627-43

[84] Deng J, Dong W, Socher R. et al. ImageNet:a large-scale hier-archical image database. In: IEEE-Computer-Society Conference on Computer Vision and Pattern Recognition Workshops. Miami Beach, FL, 2009,248-55

[85] Xie E, Wang W, Yu Z. et al. SegFormer:simple and efficient design for semantic segmentation with transformers. In:35th Conference on Neural Information Processing Systems (NeurIPS). Red Hook, NY, USA: Curran Associates Inc., 2021,12077-90

[86] Huang G, Liu Z, Pleiss G. et al. Convolutional networks with dense connectivity. IEEE Trans Pattern Anal Mach Intell. 2022; 44:8704-16

[87] Hu J, Shen L, Albanie S. et al. Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell. 2020; 42:2011-23

[88] Fitzgibbon AW. Robust registration of 2D and 3D point sets. Image Vis Comput. 2003; 21:1145-53