Integrating machine learning, optical sensors, and robotics for advanced food quality assessment and food processing

Abstract

Machine learning, in combination with optical sensing, extracts key features from high-dimensional data for non-destructive food quality assessment. This approach overcomes the limitations of traditional destructive and labor-intensive methods, facilitating real-time decision-making for food quality profiling and robotic handling. This mini-review highlights various optical techniques integrated with machine learning for assessing food quality, including chemical profiling methods such as near-infrared, Raman, and hyperspectral imaging spectroscopy, as well as visual analysis such as RGB imaging. In addition, the review presents the application of robotics and computer vision techniques to assess food quality and then drives the automation of food harvesting, grading, and processing. Lastly, the review discusses current challenges and opportunities for future research.

Keywords

Food quality / Artificial intelligence / Optical sensors / Deep learning / Robotics / Automation

Author contributions

The authors confirm their contribution to the paper as follows: writing - original draft: Lee IH; writing - review & editing, visualization: Lee IH, Ma L; conceptualization, supervision, project administration, funding acquisition: Ma L. Both authors reviewed and approved the final version of the manuscript.

Data availability

Data sharing is not applicable to this mini-review article as no datasets have been generated or analyzed.

Acknowledgments

This work was financially supported by the U.S. Department of Agriculture's National Institute of Food and Agriculture (USDA-NIFA) Capacity Building Grants for Non-Land-Grant Colleges of Agriculture Program (Grant No. 2024-70001-43485) and Oregon State University Startup Grant.

Conflict of interest

The authors declare that they have no conflict of interest.

References

[1] Petrescu DC, Vermeir I, Petrescu-Mag RM. 2020. Consumer understanding of food quality, healthiness, and environmental impact: A crossnational perspective. International Journal of Environmental Research and Public Health 17:169

[2] Tian S, Xu H. 2022. Nondestructive methods for the quality assessment of fruits and vegetables considering their physical and biological variability. Food Engineering Reviews 14:380-407

[3] Kong F, Singh R. 2016. The Stability and Shelf Life of FoodChemical deterioration and physical instability of foods and beverages. In The Stability and Shelf Life of Food, ed. Subramaniam P. 2 ^nd Edition. Sawston, United Kingdom: Woodhead Publishing. pp. 43-76pp. doi: 10.1016/b978-0-08-100435-7.00002-2

[4] Su WH, He HJ, Sun DW. 2017. Non-destructive and rapid evaluation of staple foods quality by using spectroscopic techniques: a review. Critical Reviews in Food Science and Nutrition 57:1039-51

[5] Narendra VG, Hareesh KS. 2010. Prospects of computer vision automated grading and sorting systems in agricultural and food products for quality evaluation. International Journal of Computer Applications 1( 4):1-9

[6] Anderson AK. 2008. Biogenic and volatile amine-related qualities of three popular fish species sold at Kuwait fish markets. Food Chemistry 107:761-67

[7] Oraguzie N, Alspach P, Volz R, Whitworth C, Ranatunga C, et al. 2009. Postharvest assessment of fruit quality parameters in apple using both instruments and an expert panel. Postharvest Biology and Technology 52:279-87

[8] Magwaza LS, Tesfay SZ. 2015. A review of destructive and non-destructive methods for determining avocado fruit maturity. Food and Bioprocess Technology 8:1995-2011

[9] Singh CB, Jayas DS. 2013. Optical sensors and online spectroscopy for automated quality and safety inspection of food products. In Robotics and Automation in the Food Industry. UK: Woodhead Publishing. pp. 111-29. doi: 10.1533/9780857095763.1.111

[10] Magnus I, Virte M, Thienpont H, Smeesters L. 2021. Combining optical spectroscopy and machine learning to improve food classification. Food Control 130:108342

[11] Goyache F, Bahamonde A, Alonso J, Lopez S, del Coz JJ, et al. 2001. The usefulness of artificial intelligence techniques to assess subjective quality of products in the food industry. Trends in Food Science & Technology 12:370-81

[12] Tunny SS, Kurniawan H, Amanah HZ, Baek I, Kim MS, et al. 2023. Hyperspectral imaging techniques for detection of foreign materials from fresh-cut vegetables. Postharvest Biology and Technology 201:112373

[13] Tian H, Wang T, Liu Y, Qiao X, Li Y. 2020. Computer vision technology in agricultural automation-a review. Information Processing in Agriculture 7:1-19

[14] Tapia-Mendez E, Cruz-Albarran IA, Tovar-Arriaga S, Morales-Hernandez LA. 2023. Deep learning-based method for classification and ripeness assessment of fruits and vegetables. Applied Sciences 13:12504

[15] Hussein Z, Fawole OA, Opara UL. 2020. Harvest and postharvest factors affecting bruise damage of fresh fruits. Horticultural Plant Journal 6:1-13

[16] Umapathi R, Park B, Sonwal S, Rani GM, Cho Y, et al. 2022. Advances in optical-sensing strategies for the on-site detection of pesticides in agricultural foods. Trends in Food Science & Technology 119:69-89

[17] Ran W, Chen Y. 2023. Fresh produce supply chain coordination based on freshness preservation strategy. Sustainability 15:8184

[18] He M, Li C, Cai Z, Qi H, Zhou L, et al. 2024. Leafy vegetable freshness identification using hyperspectral imaging with deep learning approaches. Infrared Physics & Technology 138:105216

[19] Guo Z, Zhang Y, Wang J, Liu Y, Jayan H, et al. 2023. Detection model transfer of apple soluble solids content based on NIR spectroscopy and deep learning. Computers and Electronics in Agriculture 212:108127

[20] Mohammed M, Srinivasagan R, Alzahrani A, Alqahtani NK. 2023. Machine-learning-based spectroscopic technique for non-destructive estimation of shelf life and quality of fresh fruits packaged under modified atmospheres. Sustainability 15:12871

[21] Aboah J, Lees N. 2020. Consumers use of quality cues for meat purchase: Research trends and future pathways. Meat Science 166:108142

[22] Hassoun A, Karoui R. 2017. Quality evaluation of fish and other seafood by traditional and nondestructive instrumental methods: Advantages and limitations. Critical Reviews in Food Science and Nutrition 57:1976-98

[23] Font-I-Furnols M, Čandek-Potokar M, Maltin C, Prevolnik Povše M. 2015. A handbook of reference methods for meat quality assessment. Brussels, Belgium: European Cooperation in Science and Technology (COST).

[24] Honikel KO. 1991. Assessment of meat quality. In Animal biotechnology and the quality of meat production, eds. Fiems LO, Cottyn BG, Demeyer DI. Amsterdam: Elsevier. pp. 107-25. doi: 10.1016/b978-0-444-88930-0.50013-4

[25] Warner R. 2014. Measurement of meat quality. Measurements of waterholding capacity and color:objective and subjective. In Encyclopedia of Meat Sciences, eds. Dikeman M, Devine C. 2nd Edition. UK: Academic Press. pp. 164-71. doi: 10.1016/b978-0-12-384731-7.00210-5

[26] Damez JL, Clerjon S. 2008. Meat quality assessment using biophysical methods related to meat structure. Meat Science 80:132-49

[27] Chen Q, Xie Y, Yu H, Guo Y, Yao W. 2023. Non-destructive prediction of colour and water-related properties of frozen/thawed beef meat by Raman spectroscopy coupled multivariate calibration. Food Chemistry 413:135513

[28] Meenu M, Kurade C, Neelapu BC, Kalra S, Ramaswamy HS, et al. 2021. A concise review on food quality assessment using digital image processing. Trends in Food Science & Technology 118:106-24

[29] Cai J, Lu Y, Olaniyi E, Wang S, Dahlgren C, et al. 2024. Beef marbling assessment by structured-illumination reflectance imaging with deep learning. Journal of Food Engineering 369:111936

[30] Lu Y, Lu R. 2019. Structured-illumination reflectance imaging for the detection of defects in fruit: Analysis of resolution, contrast and depthresolving features. Biosystems Engineering 180:1-15

[31] Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. 2018. Unet++: a nested u-net architecture for medical image segmentation. Proc. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4^th International Workshop, DLMIA 2018, and 8^th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018. Cham:Springer. pp. 3Cham11. doi: 10.1007/978-3-030-00889-5_1

[32] Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H. 2018. Encoderdecoder with atrous separable convolution for semantic image segmentation. Proc. Proceedings of the European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, 2018. Cham: Springer. pp. 801-18. doi: 10.1007/978-3-030-00889-5_1

[33] Xie E, Wang W, Yu Z, Anandkumar A, Alvarez JM, et al. 2021. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34:12077-90

[34] Cheng JH, Sun DW, Pu H, Zhu Z. 2015. Development of hyperspectral imaging coupled with chemometric analysis to monitor K value for evaluation of chemical spoilage in fish fillets. Food Chemistry 185:245-53

[35] Jinadasa B. 2014. Determination of quality of marine fishes based on total volatile base nitrogen test (TVB-N). Nature and Science 12( 5):106-11

[36] Wang K, Yue Z, Lin H, Wang Q, Wang L, et al. 2023. Rapid classification of the freshness grades of sea bass (Lateolabrax japonicus) fillets using a portable Raman spectrometer with machine learning method. Microchemical Journal 192:108948

[37] Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 October 2017, Venice, Italy. pp. 618-26. doi: 10.1109/ICCV.2017.74

[38] Yildiz MB, Yasin ET, Koklu M. 2024. Fisheye freshness detection using common deep learning algorithms and machine learning methods with a developed mobile application. European Food Research and Technology 250:1919-32

[39] landola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, et al. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv Preprint

[40] Simonyan K, Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv Preprint

[41] Nayik GA, Muzaffar K, Gull A. 2015. Robotics and food technology: a mini review. Journal of Nutrition & Food Sciences 5:4

[42] Tituaña L, Gholami A, He Z, Xu Y, Karkee M, et al. 2024. A small autonomous field robot for strawberry harvesting. Smart Agricultural Technology 8:100454

[43] Diwan T, Anirudh G, Tembhurne JV. 2023. Object detection using YOLO: Challenges, architectural successors, datasets and applications. Multimedia Tools and Applications 82:9243-75

[44] Zhang K, Lammers K, Chu P, Li Z, Lu R. 2024. An automated apple harvesting robot-from system design to field evaluation. Journal of Field Robotics 41:2384-400

[45] Lu Z, Zhao M, Luo J, Wang G, Wang D. 2021. Design of a winter-jujube grading robot based on machine vision. Computers and Electronics in Agriculture 186:106170

[46] Aly BA, Low T, Long D, Brett P, Baillie C. 2024. Tactile sensing for tissue discrimination in robotic meat cutting: a feasibility study. Journal of Food Engineering 363:111754

[47] Manko M, Smolkin O, Romanov D, de Medeiros Esper I, Popov A, et al. 2024. Deep learning model for automatic limb detection and gripping in a novel meat factory cell. Smart Agricultural Technology 8:100486

[48] de Medeiros Esper I, Gangsei LE, Cordova-Lopez LE, Romanov D, Bjørnstad PH, et al. 2024. 3D model based adaptive cutting system for the meat factory cell: overcoming natural variability. Smart Agricultural Technology 7:100388

[49] Raheem D, Treiblmaier H, Mohammed WM, Ferrer BR, Martinez-Lastra JL. 2024. Robotics as key enabler technology in Food Industry 4.0 and beyond. In Food Industry 4.0, ed. Hassoun A. UK: Academic Press. pp. 121-31. doi: 10.1016/b978-0-443-15516-1.00007-4

[50] Wright R, Parekh S, White R, Losey DP. 2024. Safely and autonomously cutting meat with a collaborative robot arm. Scientific Reports 14:299

[51] Yang L, Shami A. 2020. On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing 415:295-316