Pears classification by identifying internal defects based on X-ray images and neural networks

Abstract

In order to increase the sales and profitability, it is essential to classify the pears according to the external morphology (including shape, color and luster) and internal defects that can be quantitatively detected by various approaches. However, the existing classification methods concentrate mainly on the external quality rather than the internal defects. Therefore, this investigation develops an efficient and accurate classification method that can identify the internal sclerosis and bruises by combining the X-ray non-destructive testing and the convolutional neural network. Initially, the relations between the characteristics of the internal defects, i.e., internal sclerosis and bruises, and the grayscale features of the X-ray images are analyzed to provide the experimental data and demonstrate the theoretical foundations. Then, the X-ray images are processed by resolution reduction, feature enhancement and gradient reconstruction to improve the training efficiency and classification precision. Finally, the 18-layer residual network (ResNet-18) is optimized and trained to identify the internal bruises and sclerosis and classify the pears based on the identification results. It is found that the overall accuracy can reach 96.67% for identifying the bruised and sclerotic pears. The proposed method could also be applied to other fruits for defects identification and quality classification.

Keywords

Pears classification / Internal defects / X-ray images / Residual network

References

[1] PholphoT, PathaveeratS, SirisomboonP. Classification of longan fruit bruising using visible spectroscopy. J Food Eng, 2011, 104(1): 169-172.

[2] OparaUL, PatharePB. Bruise damage measurement and analysis of fresh horticultural produce-a review. Postharvest Biol Technol, 2014, 91: 9-24.

[3] DoerflingerFC, RickardBJ, NockJF, et al.. An economic analysis of harvest timing to manage the physiological storage disorder firm flesh browning in 'empire' apples. Postharvest Biol Technol, 2015, 107: 1-8.

[4] ZengX, MiaoY, UbaidS, et al.. Detection and classification of bruises of pears based on thermal images. Postharvest Biol Technol, 2020, 161. 111090

[5] BaranowskiP, LipeckiJ, MazurekW, et al.. Detection of watercore in 'gloster' apples using thermography. Postharvest Biol Technol, 2008, 47(3): 358-366.

[6] HuangY, LuR, ChenK. Detection of internal defect of apples by a multichannel VIS/NIR spectroscopic system. Postharvest Biol Techno, 2020, 161. 111065

[7] HanD, TuR, LuC, et al.. Nondestructive detection of brown core in the chinese pear 'yali' by transmission visible-NIR spectroscopy. Food Control, 2006, 17: 604-608.

[8] KhatiwadaBP, SubediPP, HayesC, et al.. Assessment of internal flesh browning in intact apple using visible-short wave near infrared spectroscopy. Postharvest Biol Technol, 2016, 120: 103-111.

[9] MogollónR, ContrerasC, Da Silva NetaML, et al.. Non-destructive prediction and detection of internal physiological disorders in 'keitt' mango using a hand-held VIS-NIR spectrometer. Postharvest Biol Technol, 2020, 167. 111251

[10] HerremansE, Melado-HerrerosA, DefraeyeT, et al.. Comparison of X-ray CT and MRI of watercore disorder of different apple cultivars. Postharvest Biol Technol, 2014, 87: 42-50.

[11] Van DaelM, VerbovenP, ZanellaA, et al.. Combination of shape and X-ray inspection for apple internal quality control: in silico analysis of the methodology based on X-ray computed tomography. Postharvest Biol Technol, 2019, 148: 218-227.

[12] Van De LooverboschT, RaeymaekersE, VerbovenP, et al.. Non-destructive internal disorder detection of conference pears by semantic segmentation of X-ray CT scans using deep learning. Expert Syst Appl, 2021, 176. 114925

[13] MuziriT, TheronKI, CantreD, et al.. Microstructure analysis and detection of mealiness in 'forelle' pear (Pyrus communis L.) by means of X-ray computed tomography. Postharvest Biol Technol, 2016, 120: 145-156.

[14] MagwazaLS, OparaUL. Investigating non-destructive quantification and characterization of pomegranate fruit internal structure using X-ray computed tomography. Postharvest Biol Technol, 2014, 95: 1-6.

[15] ShahinMA, TollnerEW, McclendonRW. AE-automation and emerging technologies: artificial intelligence classifiers for sorting apples based on watercore. J Agric Eng Res, 2001, 79: 265-274.

[16] MatsuiT, KamataT, KosekiS, et al.. Development of automatic detection model for stem-end rots of 'hass' avocado fruit using X-ray imaging and image processing. Postharvest Biol Technol, 2022, 192. 111996

[17] KotwaliwaleN, WecklerPR, BrusewitzGH, et al.. Non-destructive quality determination of pecans using soft X-rays. Postharvest Biol Technol, 2007, 45: 372-380.

[18] HaffRP, SlaughterDC, SarigY, et al.. X-ray assessment of translucency in pineapple. J Food Process Preserv, 2006, 30: 527-533.

[19] LiJ, RaoX, WangF, et al.. Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods. Postharvest Biol Technol, 2013, 82: 59-69.

[20] ZhangX, ZhuY, SuY, et al.. Quantitative extraction and analysis of pear fruit spot phenotypes based on image recognition. Comput Electron Agric, 2021, 190. 106474

[21] WangB, YinJ, LiuJ, et al.. Extraction and classification of apple defects under uneven illumination based on machine vision. J Food Process Eng, 2022, 4513976.

[22] ZhangY, DongZ, ChenX, et al.. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation. Multimed Tools Appl, 2019, 78: 3613-3632.

[23] AlhudhaifA, PolatK, KaramanO. Determination of COVID-19 pneumonia based on generalized convolutional neural network model from chest X-ray images. Expert Syst Appl, 2021, 180. 115141

[24] YasakaK, AkaiH, KunimatsuA, et al.. Deep learning with convolutional neural network in radiology. Jpn J Radiol, 2018, 36: 257-272.

[25] KhanE, RehmanMZU, AhmedF, et al.. Chest X-ray classification for the detection of COVID-19 using deep learning techniques. Sensors, 2022, 221211.

[26] LuY, LuR, ZhangZ. Detection of subsurface bruising in fresh pickling cucumbers using structured-illumination reflectance imaging. Postharvest Biol Technol, 2021, 180. 111624

[27] YuX, LuH, WuD. Development of deep learning method for predicting firmness and soluble solid content of postharvest korla fragrant pear using VIS/NIR hyperspectral reflectance imaging. Postharvest Biol Technol, 2018, 141: 39-49.

[28] BobelynE, SerbanA, NicuM, LammertynJ, NicolaiBM, SaeysW. Postharvest quality of apple predicted by NIR-spectroscopy: study of the effect of biological variability on spectra and model performance. Postharvest Biol Technol, 2010, 55: 133-143.

[29] NicolaïBM, BeullensK, BobelynE, et al.. Nondestructive measurement of fruit and vegetable quality by means of NIR spectroscopy: a review. Postharvest Biol Technol, 2007, 46: 99-118.

[30] ColnagoLA, AndradeFD, SouzaAA, et al.. Why is inline NMR rarely used as industrial sensor? Challenges and opportunities. Chem Eng Technol, 2014, 37: 191-203.

[31] YangW, ZhangJ, WangH, et al.. Peroxisome proliferator-activated receptor γ regulates angiotensin II-induced catalase downregulation in adventitial fibroblasts of rats. FEBS Lett, 2011, 585: 761-766.

[32] NguyenHD, CaiR, ZhaoH, et al.. Towards more efficient security inspection via deep learning: a task-driven X-ray image cropping scheme. Micromachines, 2022, 13565.

[33] Van De LooverboschT, Rahman BhuiyanMH, VerbovenP, et al.. Nondestructive internal quality inspection of pear fruit by X-ray CT using machine learning. Food Control, 2020, 113. 107170

[34] Vélez RiveraN, Gómez-SanchisJ, Chanona-PérezJ, et al.. Early detection of mechanical damage in mango using NIR hyperspectral images and machine learning. Biosyst Eng, 2014, 122: 91-98.

[35] WeiH, GuY. A machine learning method for the detection of brown core in the Chinese pear variety huangguan using a MOS-based e-nose. Sensors, 2020, 204499.

Funding

National Natural Science Foundation of China(12102120)

Natural Science Foundation of Hebei Province(A2021202019)

Department of Education of Hebei Province(ZD2021029)

Tianjin Municipal Science and Technology Program(22JCZDJC00070)

RIGHTS & PERMISSIONS

Shanghai University and Periodicals Agency of Shanghai University and Springer-Verlag GmbH Germany, part of Springer Nature