Back Propagation Neural Network model for analysis of hyperspectral images to predict apple firmness

Abstract

The potential of employing hyperspectral imaging (HSI) in the near-infrared (NIR) range ( 386.82-1,004.50 nm ) for predicting the firmness of 'Fuji' apples cultivated in Aksu has been evaluated. The performance of seven preprocessing algorithms and two feature selection algorithms was evaluated. The coefficient of determination ( R² ) and root mean square error (RMSE) of Partial Least Squares (PLS) models are contrasted using various inputs. These results confirm that the Multiplicative Scatter Correction (MSC) preprocessing algorithm was the optimal choice ( ${R}_{p}^{2}=0.7925,RMSEP=0.6537$ ), and the Competitive Adaptive Reweighted Sampling (CARS) feature selection algorithm demonstrated superior performance $\left({R}_{p}^{2}=0.8325,RMSEP=0.6257\right.$ ). Based on the aforementioned findings, PLS, Multiple Linear Regression (MLR), Heterogeneous Transfer Learning (HTL), and Back Propagation Neural Network (BPNN) models were constructed for cross-validation purposes. The experimental results indicate that the CARS-BPNN model exhibits the optimal prediction performance, with an ${R}_{p}^{2}$ value of 0.9350 and an RMSEP value of 0.4654. The results of the research indicated that a deep learning method combined with hyperspectral imaging technology could be utilized to non-destructively detect the firmness of 'Fuji' apples, which will be beneficial and potentially applicable for post-harvest fruit firmness monitoring. This research provides a reference point for the non-destructive detection of apple in the selection of preprocessing, feature selection algorithms, and predicting firmness model.

Keywords

Non-destructive detection / Deep learning / 'Fuji' apple / Hyperspectral image / Feature selection

Author contributions

The authors confirm contribution to the paper as follows: conceptualization: Gao X, Ban Z; project administration: Ban Z; writing original draft: Li S; writing - review & editing: Li S, Chen Y, Zhang X, Ban Z, Chen C; methodology: Li S, Chen Y, Wang J, Jiang Y, Chen C; investigation: Li S, Zhang X, Jiang Y; formal analysis: Li S, Wang J, Jiang Y, Ban Z; data curation: Li S, Chen Y, Gao X; software: Zhang X, Wang J, Gao X, Jiang Y; resources, supervision: Chen C. All authors reviewed the results and approved the final version of the manuscript.

Data availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

Acknowledgments

This work was supported by 'Pioneer' and 'Leading Goose' Research and Development Plan Project of Zhejiang Province (2022C04039), Major Scientific Research Achievement Transformation Project of Ningxia Hui Autonomous Region (2023CJE09060), Tianjin Science and Technology Program Project (22ZYCGSN00170, 22ZYCGSN00470), and International Exchanges Funds offered by the Royal Society (No. IEC\NSFC\233076).

Conflict of interest

The authors declare that they have no conflict of interest.

References

[1] Wang J, Liu T. 2022. Spatiotemporal evolution and suitability of apple production in China from climate change and land use transfer perspectives. Food and Energy Security 11:e386

[2] Oyenihi AB, Belay ZA, Mditshwa A, Caleb OJ. 2022. "An apple a day keeps the doctor away": The potentials of apple bioactive constituents for chronic disease prevention. Journal of Food Science 87:2291-309

[3] Harker FR, Roigard CM, Colonna AE, Jin D, Ryan G, et al. 2024. The relative importance of postharvest eating quality and sustainability attributes for apple fruit: A case study using new sensory-consumer approaches. Postharvest Biology and Technology 217:113099

[4] Cheng XY, Zhao YY, Xue HL, Bi Y, Sun CC, et al. 2022. Model fit based on the weight loss and texture parameters of MAP cherry tomatoes during storage. Journal of Food Processing and Preservation 46:e16204

[5] Bejaei M, Stanich K, Cliff MA. 2021. Modelling and classification of apple textural attributes using sensory, instrumental and compositional analyses. Foods 10:384-97

[6] Yang X, Zhu L, Huang X, Zhang Q, Li S, et al. 2022. Determination of the soluble solids content in korla fragrant pears based on visible and nearinfrared spectroscopy combined with model analysis and variable selection. Frontiers in Plant Science 13:938162

[7] Liu Y, Wu Q, Huang J, Zhang X, Zhu Y, et al. 2021. Comparison of apple firmness prediction models based on non-destructive acoustic signal. International Journal of Food Science and Technology 56:6443-50

[8] Martínez Gila DM, Navarro Soto JP, Satorres Martínez S, Gómez Ortega J, Gámez García J. 2022. The advantage of multispectral images in fruit quality control for extra virgin olive oil production. Food Analytical Methods 15:75-84

[9] Wang C, Liu B, Liu L, Zhu Y, Hou J, et al. 2021. A review of deep learning used in the hyperspectral image analysis for agriculture. Artificial Intelligence Review 54:5205-53

[10] Wesoły M, Przewodowski W, Ciosek-Skibińska P. 2023. Electronic noses and electronic tongues for the agricultural purposes. Trends in Analytical Chemistry 164:117082-103

[11] Schlie TP, Dierend W, Köpcke D, Rath T. 2022. Detecting low-oxygen stress of stored apples using chlorophyll fluorescence imaging and histogram division. Postharvest Biology and Technology 189:111901-09

[12] Peng K, Ma W, Lu J, Tian Z, Yang Z. 2023. Application of machine vision technology in citrus production. Applied Sciences 13:9334

[13] Zhang P, Wang H, Ji H, Li Y, Zhang X, et al. 2023. Hyperspectral imagingbased early damage degree representation of apple: a method of correlation coefficient. Postharvest Biology and Technology 199:112309-17

[14] Arendse E, Nieuwoudt H, Magwaza LS, Nturambirwe JFI, Fawole OA, et al. 2021. Recent advancements on vibrational spectroscopic techniques for the detection of authenticity and adulteration in horticultural products with a specific focus on oils, juices and powders. Food and Bioprocess Technology 14:1-22

[15] Shlezinger N, Whang J, Eldar YC, Dimakis AG. 2023. Model based deep learning. Proceedings of the IEEE 111:465-99

[16] Zhou L, Zhang C, Liu F, Qiu Z, He Y. 2019. Application of deep learning in food: A review. Comprehensive Reviews in Food Science and Food Safety 18:1793-811

[17] Li S, Song Q, Liu Y, Zeng T, Liu S, et al. 2023. Hyperspectral imagingbased detection of soluble solids content of loquat from a small sample. Postharvest Biology and Technology 204:112454

[18] Tian Y, Sun J, Zhou X, Yao K, Tang N. 2022. Detection of soluble solid content in apples based on hyperspectral technology combined with deep learning algorithm. Journal of Food Processing and Preservation 46:e16414

[19] Ma T, Xia Y, Inagaki T, Tsuchikawa S. 2021. Non-destructive and fast method of mapping the distribution of the soluble solids content and pH in kiwifruit using object rotation near-infrared hyperspectral imaging approach. Postharvest Biology and Technology 174:111440-47

[20] Park B, Shin T, Cho JS, Lim JH, Park KJ. 2023. Improving blueberry firmness classification with spectral and textural features of microstructures using hyperspectral microscope imaging and deep learning. Postharvest Biology and Technology 195:112154-64

[21] Ragavendra S, Ganguli S, Selvan PT, Nayak MM, Chaudhury S, et al. 2022. Deep learning based dual channel banana grading system using convolution neural network. Journal of Food Quality 2022:6050284

[22] Xiang Y, Chen Q, Su Z, Zhang L, Chen Z, et al. 2022. Deep learning and hyperspectral images based tomato soluble solids content and firmness estimation. Frontiers in Plant Science 13:860656-66

[23] Xu M, Sun J, Yao K, Cai Q, Shen J, et al. 2022. Developing deep learning based regression approaches for prediction of firmness and pH in kyoho grape using Vis/NIR hyperspectral imaging. Infrared Physics and Technology 120:104003-12

[24] Liu P, Zhang P, Ni F, Hu Y. 2021. Feasibility of nondestructive detection of apple crispness based on spectroscopy and machine vision. Journal of Food Process Engineering 44:13802

[25] Shao Y, Ji S, Xuan G, Wang K, Xu L, et al. 2024. Soluble solids content monitoring and shelf life analysis of winter jujube at different maturity stages by Vis-NIR hyperspectral imaging. Postharvest Biology and Technology 210:112773-80

[26] Kim HJ, Baek JW, Chung K. 2021. Associative knowledge graph using fuzzy clustering and min-max normalization in video contents. IEEE Access 9:74802-16

[27] Siino M, Tinnirello I, La Cascia M. 2024. Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on transformers and traditional classifiers. Information Systems 121:102342

[28] Liu HL, Yu CH, Wan LC, Qin SJ, Gao F, et al. 2022. Quantum mean centering for block-encoding-based quantum algorithm. Physica A: Statistical Mechanics and its Applications 607:128227

[29] Sohn SI, Pandian S, Oh YJ, Zaukuu JLZ, Na CS, et al. 2022. Vis-NIR spectroscopy and machine learning methods for the discrimination of transgenic Brassica napus L. and their hybrids with B. juncea. Processes 10:240

[30] Butt UM, Letchmunan S, Ali M, Hassan FH, Baqir A, et al. 2021. Machine learning based diabetes classification and prediction for healthcare applications. Journal of Healthcare Engineering 2021:9930985

[31] Figueiredo NS, Ferreira LHC, Dutra OO. 2019. An approach to savitzkygolay differentiators. Circuits Systems and Signal Processing 38:4369-79

[32] Endut R, Sabri MSA, Aljunid SA, Ali N, Laili AR, et al. 2023. Prediction of potassium (K) content in soil analysis utilizing near-infrared (NIR) spectroscopy. Journal of Advanced Research in Applied Sciences and Engineering Technology 33:92-101

[33] Gao Q, Wang P, Niu T, He D, Wang M, et al. 2022. Soluble solid content and firmness index assessment and maturity discrimination of Malus micromalus Makino based on near-infrared hyperspectral imaging. Food Chemistry 370:131013

[34] Xuan W, Wang Y. 2021. Competitive adaptive reweighted sampling method for fault detection. Journal of Physics: Conference Series 1820:012078

[35] Sun J, Yang W, Feng M, Liu Q, Kubar M. 2020. An efficient variable selection method based on random frog for the multivariate calibration of NIR spectra. RSC Advances 10:16245-53

[36] Meng Q, Shang J, Huang R, Zhang Y. 2021. Determination of soluble solids content and firmness in plum using hyperspectral imaging and chemometric algorithms. Journal of Food Process Engineering 44:e13597

[37] Chen Y, Jiang X, Liu Q, Wei Y, Wang F, et al. 2024. A hyperspectral imaging technique for rapid non-destructive detection of soluble solid content and firmness of wolfberry. Journal of Food Measurement and Characterization 18:7927-41

[38] Tang N, Sun J, Yao K, Zhou X, Tian Y, et al. 2021. Identification of Lycium barbarum varieties based on hyperspectral imaging technique and competitive adaptive reweighted sampling-whale optimization algo-rithm-support vector machine. Journal of Food Process Engineering 44:e13603

[39] Xia Z, Yang J, Wang J, Wang S, Liu Y. 2020. Optimizing rice near-infrared models using fractional order savitzky-golay derivation (FOSGD) combined with competitive adaptive reweighted sampling (CARS). Applied Spectroscopy 74:417-26

[40] Meng Q, Tan T, Feng S, Wen Q, Shang J. 2024. Prediction and visualization map for physicochemical indices of kiwifruits by hyperspectral imaging. Frontiers in Nutrition 11:1364274

[41] Xing Z, Du C, Shen Y, Ma F, Zhou J. 2021. A method combining FTIR-ATR and raman spectroscopy to determine soil organic matter: Improvement of prediction accuracy using competitive adaptive reweighted sampling (CARS). Computers and Electronics in Agriculture 191:106549

[42] Yang B, Li X, Wu L, Chen Y, Zhong F, et al. 2022. Citrus huanglongbing detection and semi-quantification of the carbohydrate concentration based on micro-FTIR spectroscopy. Analytical and Bioanalytical Chemistry 414:6881-97

[43] Chen S, Lou F, Tuo Y, Tan S, Peng K, et al. 2023. Prediction of soil water content based on hyperspectral reflectance combined with competitive adaptive reweighted sampling and random frog feature extraction and the back-propagation artificial neural network method. Water 15:2726

[44] Gorzelany J, Belcar J, Kuźniar P, Niedbała G, Pentoś K. 2022. Modelling of mechanical properties of fresh and stored fruit of large cranberry using multiple linear regression and machine learning. Agriculture 12:200

[45] Luo Y, Wen Y, Liu T, Tao D. 2019. Transferring knowledge fragments for learning distance metric from a heterogeneous domain. IEEE Transactions on Pattern Analysis and Machine Intelligence 41:1013-26

[46] Elsherbiny O, Fan Y, Zhou L, Qiu Z. 2021. Fusion of feature selection methods and regression algorithms for predicting the canopy water content of rice based on hyperspectral data. Agriculture 11:51-71

[47] Ciccoritti R, Paliotta M, Amoriello T, Carbone K. 2019. FT-NIR spectroscopy and multivariate classification strategies for the postharvest quality of green-fleshed kiwifruit varieties. Scientia Horticulturae 257:108622-31