Multimodal fusion of UAV-based computer vision and plant water content dynamics for high-throughput soybean maturity classification
Yaxin Li , Tingting Li , Yanqiang Zhao , Kunpeng Jiang , Yuying Ye , Shuai Wang , Zhipeng Zhou , Qiaorong Wei , Rongsheng Zhu , Qingshan Chen , Limin Hu , Mingliang Yang , Le Xu
Crop and Environment ›› 2025, Vol. 4 ›› Issue (4) : 241 -255.
Multimodal fusion of UAV-based computer vision and plant water content dynamics for high-throughput soybean maturity classification
Soybean is the most important oilseed and forage crop globally. Advancements in high-throughput phenotyping technologies are critical for accelerating genetic improvement in modern breeding research. However, conventional methods for assessing soybean maturity remain labor intensive. This study developed high-throughput phenotyping algorithms based on unmanned aerial vehicle (UAV) multispectral imagery combined with machine learning to monitor the maturity process of 30 soybean cultivars in large-scale breeding trials. UAV images and plant water content (PWC) data were collected to classify soybean maturity into four distinct phases: immaturity (i.e. the period before R5 stage), late pod filling (i.e. R5 to R6), physiological maturity (i.e. R7), and harvesting maturity (i.e. R8). We evaluated the performance of three classification approaches: (1) a computer vision model utilizing UAV-derived color features, (2) a PWC-based model retrieving PWC dynamics using UAV-derived feature, and (3) a multimodal fusion model integrating computer vision and PWC dynamics. Computer vision model effectively distinguished immature and mature plants but showed limitations in resolving specific maturity phases due to genetic variation in canopy color among cultivars (training set accuracy: 0.71; validation set accuracy: 0.69). The sensitive UAV-derived features were applied to establish the prediction model of PWC using convolutional neural network, which achieved the highest R2 (training set: R2 = 0.95; validation set: R2 = 0.86) between the predicted and measured PWC. The PWC-based algorithm outperformed the computer vision approach, achieving higher classification accuracy (training set: 0.78; validation set: 0.79). Strong correlations between PWC and pod water content, stem water content, and leaf water content underscored the physiological relevance of PWC in tracking maturation dynamics. Further improvement in classification accuracy was achieved with the multimodal fusion model (training set: 0.84; validation set: 0.83), which combined the information of computer vision and PWC dynamics. It was also confirmed that the multimodal fusion model achieved the lowest misclassification rate in the validation analysis across diverse soybean cultivars. These findings emphasize the potential of integrating UAV-based computer vision and PWC features to improve the accuracy and efficiency of soybean maturity classification. The proposed multimodal approach offers a robust framework for phenotypic selection and trait evaluation, providing valuable insights for soybean breeding programs.
High-throughput phenotyping / Maturity classification / Multimodal fusion / Plant water content / Soybean / UAV
| [1] |
|
| [2] |
|
| [3] |
|
| [4] |
|
| [5] |
|
| [6] |
|
| [7] |
|
| [8] |
|
| [9] |
|
| [10] |
|
| [11] |
|
| [12] |
|
| [13] |
|
| [14] |
|
| [15] |
|
| [16] |
|
| [17] |
|
| [18] |
|
| [19] |
|
| [20] |
|
| [21] |
|
| [22] |
|
| [23] |
|
| [24] |
|
| [25] |
|
| [26] |
|
| [27] |
|
| [28] |
|
| [29] |
|
| [30] |
|
| [31] |
|
| [32] |
|
| [33] |
|
| [34] |
|
| [35] |
|
| [36] |
|
| [37] |
|
| [38] |
|
| [39] |
|
| [40] |
|
| [41] |
|
| [42] |
|
| [43] |
|
| [44] |
|
| [45] |
|
| [46] |
|
| [47] |
|
| [48] |
|
| [49] |
|
| [50] |
|
| [51] |
NSBC (National Bureau of Statistics of China), 2023.China Statistical Yearbook. China Statistics Press, Beijing. |
| [52] |
|
| [53] |
|
| [54] |
|
| [55] |
|
| [56] |
|
| [57] |
|
| [58] |
|
| [59] |
|
| [60] |
|
| [61] |
|
| [62] |
|
| [63] |
|
| [64] |
|
| [65] |
|
| [66] |
|
| [67] |
|
| [68] |
|
| [69] |
|
| [70] |
|
| [71] |
|
| [72] |
|
| [73] |
|
| [74] |
|
| [75] |
|
| [76] |
|
| [77] |
|
| [78] |
|
| [79] |
|
| [80] |
|
| [81] |
|
| [82] |
|
| [83] |
|
| [84] |
|
| [85] |
|
| [86] |
|
| [87] |
|
/
| 〈 |
|
〉 |