When to use machine learning? And which problems stand to benefit?

Abstract

Despite the continued success of machine learning (ML), its indiscriminate use may not always yield the expected benefits, can waste resources, introduce unforeseen complexities, and even lead to failure. From this lens, this paper systematically addresses the following elemental questions: When should we integrate ML, and which problem-solving workflows benefit the most? Thus, this paper critically examines existing decision making frameworks and guidelines to identify their strengths and limitations in defining ML applicability. Then, through a comprehensive literature review and analysis, we outline explicit suitability criteria to assess when ML should be favored over traditional engineering methods. Based on this analysis, we propose a conceptual structured decision making framework that incorporates practical steps and checklists to guide engineers in method selection, model development, validation, and deployment. Additionally, this paper highlights critical challenges and potential future research questions.

Keywords

Engineering / Artificial intelligence / Prediction

References

[1] Verhagen WJC, Bermell-Garcia P, Van Dijk REC, Curran R. A critical review of knowledge-based engineering: an identification of research challenges. Adv Eng Inform, 2012

[2] Suh NP. Complexity in engineering. CIRP Ann, 2005

[3] Naser MZ. Machine learning for civil and environmental engineers: a practical approach to data-driven analysis, explainability, and causality, 2023, New Jersey, Wiley

[4] Carbone MR. When not to use machine learning: A perspective on potential and limitations. MRS Bull, 2022, 47: 968-974

[5] Makridakis S, Spiliotis E, Assimakopoulos V. Statistical and machine learning forecasting methods: concerns and ways forward. PLoS ONE, 2018

[6] Na C, Oh G, Song J, Park H. Do machine learning methods outperform traditional statistical models in crime prediction? A comparison between logistic regression and neural networks. Korean J Policy Stud, 2021

[7] Nasejje JB, Whata A, Chimedza C. Statistical approaches to identifying significant differences in predictive performance between machine learning and classical statistical models for survival data. PLoS ONE, 2022

[8] Reich Y, Barai SV. Evaluating machine learning models for engineering problems. Artif Intell Eng, 1999

[9] Xu Z, Saleh JH. Machine learning for reliability engineering and safety applications: review of current status and future opportunities. Reliab Eng Syst Saf, 2021

[10] Möller B, Beer M. Engineering computation under uncertainty - capabilities of non-traditional models. Comput Struct, 2008

[11] Brodnik NR, Muir C, Tulshibagwale N, Rossin J, Echlin MP, Hamel CM, Kramer SLB, Pollock TM, Kiser JD, Smith C, Daly SH. Perspective: Machine learning in experimental solid mechanics. J Mech Phys Solids, 2023

[12] Abolhasani M, Brown KA. Role of AI in experimental materials science. MRS Bull, 2023, 48: 134-141

[13] Smith JJ, Amershi S, Barocas S, Wallach H, Wortman Vaughan J (2022) REAL ML: recognizing, exploring, and articulating limitations of machine learning research. In: ACM Int. Conf. Proceeding Ser. https://doi.org/10.1145/3531146.3533122

[14] Malik MM (2020) A hierarchy of limitations in machine learning. https://arxiv.org/pdf/2002.05193. Accessed 18 May 2025

[15] Pérez-Ruiz JL, Loboda I, González-Castillo I, Pineda-Molina VM, Rendón-Cortés KA, Miró-Zárate LA. A comparative study of data-driven and physics-based gas turbine fault recognition approaches. Proc Inst Mech Eng Pt O J Risk Reliab, 2021

[16] Thompson NC, Greenewald K, Lee K, Manso GF (2020) The computational limits of deep learning. arXiv preprint arXiv:2007.05558. https://doi.org/10.48550/arXiv.2007.05558

[17] Naser MZ. Do we need exotic models? Engineering metrics to enable green machine learning from tackling accuracy-energy trade-offs. J Clean Prod, 2023, 382: 135334

[18] Radhakrishnan J, Chattopadhyay M. Determinants and Barriers of Artificial Intelligence Adoption – A Literature Review. IFIP Advances in Information and Communication Technology, 2020, 617: 89-99

[19] Hall A, Agarwal V. Barriers to adopting artificial intelligence and machine learning technologies in nuclear power. Prog Nucl Energy, 2024, 175: 105295

[20] O’Donovan P, Leahy K, Bruton K, O’Sullivan DTJ. An industrial big data pipeline for data-driven analytics maintenance applications in large-scale smart manufacturing facilities. J Big Data, 2015

[21] Lai KK. An Integrated Data Preparation Scheme for Neural Network Data Analysis. IEEE Trans Knowl Data Eng, 2006

[22] Dosilovic FK, Brcic M, Hlupic N (2018) Explainable artificial intelligence: a survey. In: 2018 41st Int. Conv. Inf Commun Technol Electron Microelectron. MIPRO 2018–Proc. https://doi.org/10.23919/MIPRO.2018.8400040

[23] Angelov PP, Soares EA, Jiang R, Arnold NI, Atkinson PM. Explainable artificial intelligence: an analytical review. WIREs Data Min Knowl Discov, 2021

[24] Bzdok D, Altman N, Krzywinski M. Statistics versus machine learning. Nat Methods, 2018

[25] Dangeti P (2017) Statistics for machine learning. https://www.geeksforgeeks.org/machine-learning/statistics-for-machine-learning/. Accessed 11 Sep 2025

[26] Boulesteix AL, Schmid M. Machine learning versus statistical modeling. Biom J, 2014

[27] Sugiyama M (2015) Introduction to statistical machine learning. https://doi.org/10.1016/C2014-0-01992-2

[28] Karapiperis K, Kochmann DM (2023) Prediction and control of fracture paths in disordered architected materials using graph neural networks. Commun Eng 2(1):32. https://doi.org/10.1038/S44172-023-00085-0

[29] Perera R, Agrawal V. Dynamic and adaptive mesh-based graph neural network framework for simulating displacement and crack fields in phase field models. Mech Mater, 2023

[30] Jakhar K, Guan Y, Mojgani R, Chattopadhyay A, Hassanzadeh P. Learning closed-form equations for subgrid-scale closures from high-fidelity data: promises and challenges. J Adv Model Earth Syst, 2024, 16: e2023MS003874

[31] Freedman LP, Cypess RH (2019) Research reproducibility: the case for standards, handbook of biomarkers and precision medicine. pp 579–595. https://doi.org/10.1201/9780429202872-65

[32] Ravindra MK, Galambos TV. Load and resistance factor design for steel. J Struct Div, 1978

[33] Naser MZ (2021) Demystifying ten big ideas and rules every fire scientist & engineer should know about blackbox, whitebox & causal artificial intelligence. https://arxiv.org/abs/2111.13756v1. Accessed 26 Jan 2022

[34] Hurvich CM, Tsai CL. Regression and time series model selection in small samples. Biometrika, 1989

[35] Geenens G. Curse of dimensionality and related issues in nonparametric functional regression. Statistics Surveys, 2011

[36] Hao N, Zhang HH. A note on high-dimensional linear regression with interactions. Am Stat, 2017

[37] Domingos P. The role of Occam’s razor in knowledge discovery. Data Min Knowl Discov, 1999

[38] Carleo G, Cirac I, Cranmer K, Daudet L, Schuld M, Tishby N, Vogt-Maranto L, Zdeborová L. Machine learning and the physical sciences. Rev Mod Phys, 2019

[39] Edelen A, Neveu N, Frey M, Huber Y, Mayes C, Adelmann A. Machine learning for orders of magnitude speedup in multiobjective optimization of particle accelerator systems. Phys Rev Accel Beams, 2020

[40] Rebala G, Ravi A, Churiwala S (2019) Designing a machine learning system. An Introd to Mach Learn. https://doi.org/10.1007/978-3-030-15729-6_18

[41] Ferreira C, Gonçalves G. Remaining useful life prediction and challenges: a literature review on the use of machine learning methods. J Manuf Syst, 2022

[42] Marwala T (2021) Rational machines and artificial intelligence. https://doi.org/10.1016/B978-0-12-820676-8.09990-7

[43] Mei F, Chen H, Yang W, Zhai G. A hybrid physics-informed machine learning approach for time-dependent reliability assessment of electromagnetic relays. Reliab Eng Syst Saf, 2024, 252: 110385

[44] Willard J, Jia X, Xu S, Steinbach M, Kumar V. Integrating scientific knowledge with machine learning for engineering and environmental systems. ACM Comput Surv, 2022

[45] Sette S, Boullart L. Genetic programming: principles and applications. Eng Appl Artif Intell, 2001

[46] Udrescu SM, Tan A, Feng J, Neto O, Wu T, Tegmark M (2020) AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity. Advances in Neural Information Processing Systems. 2020-December. https://github.com/SJ001/AI-Feynman. Accessed 31 Oct 2022

[47] Wirth R (2000) CRISP-DM : towards a standard process model for data mining. Proceedings of the Fourth International Conference on the Practical Application of Knowledge Discovery and Data Mining

[48] Kolyshkina I, Simoff S (2021) The CRISP-ML approach to handling causality and interpretability issues in machine learning. In: Proc. - 2021 IEEE Int. Conf. Big Data, Big Data 2021. https://doi.org/10.1109/BigData52589.2021.9671754

[49] Studer S, Bui TB, Drescher C, Hanuschkin A, Winkler L, Peters S, Müller KR. Towards CRISP-ML(Q): a machine learning process model with quality assurance methodology. Mach Learn Knowl Extr, 2021

[50] Kumari K, Bhardwaj M, Sharma S. OSEMN approach for real time data analysis. Int J Eng Manage Res, 2020

[51] He J, Cheng Z, Guo B. Anomaly detection in telemetry data using a jointly optimal one-class support vector machine with dictionary learning. Reliab Eng Syst Saf, 2024, 242: 109717

[52] Bang H, Selva D (2016) IFEED: interactive feature extraction for engineering design. Proc ASME Des Eng Tech Conf. https://doi.org/10.1115/DETC201660077

[53] Liu H (2018) Feature engineering for machine learning and data analytics. https://doi.org/10.1201/9781315181080

[54] Kolltveit AB, Li J (2022) Operationalizing machine learning models - a systematic literature review. In: Proc. - Work. Softw. Eng. Responsible AI, SE4RAI 2022. https://doi.org/10.1145/3526073.3527584

[55] Testi M, Ballabio M, Frontoni E, Iannello G, Moccia S, Soda P, Vessio G. MLops: a taxonomy and a methodology. IEEE Access, 2022

[56] Jamshidi S, Nafi KW, Nikanjam A, Khomh F. Evaluating machine learning-driven intrusion detection systems in IoT: performance and energy consumption. Comput Ind Eng, 2025, 204: 111103

[57] Malerba D, Pasquadibisceglie V. Data-centric AI. J Intell Inf Syst, 2024, 62: 1493-1502

[58] Habbal A, Ali MK, Abuzaraida MA. Artificial intelligence trust, risk and security management (AI TRiSM): frameworks, applications, challenges and future research directions. Expert Syst Appl, 2024

[59] Tamburri DA (2020) Sustainable MLOps: trends and challenges. In: Proc. - 2020 22nd Int. Symp. Symb. Numer. Algorithms Sci. Comput. SYNASC 2020. https://doi.org/10.1109/SYNASC51798.2020.00015

[60] Zhang X, Chan FTS, Yan C, Bose I. Towards risk-aware artificial intelligence and machine learning systems: an overview. Decis Support Syst, 2022

[61] Lisboa PJG (2013) Interpretability in machine learning – principles and practice, lecture notes in computer science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 8256 LNAI. pp 15–21. https://doi.org/10.1007/978-3-319-03200-9_2

[62] Naser MZ. Integrating machine learning models into building codes and standards: establishing equivalence through engineering intuition and causal logic. J Struct Eng, 2024, 150: 04024039

[63] Oliveira RMA, Sant’Anna ÂMO, da Silva PHF. Explainable machine learning models for defects detection in industrial processes. Comput Ind Eng, 2024, 192: 110214

[64] Nemani V, Biggio L, Huan X, Hu Z, Fink O, Tran A, Wang Y, Zhang X, Hu C. Uncertainty quantification in machine learning for engineering design and health prognostics: a tutorial. Mech Syst Signal Process, 2023

[65] Truong HL, Truong-Huu T, Cao TD. Making distributed edge machine learning for resource-constrained communities and environments smarter: contexts and challenges. J Reliab Intell Environ, 2023, 9: 119-134

[66] DeepLearing.ai, The State of Enterprise AI in 2020, DeepLearing.Ai. (2019) https://www.deeplearning.ai/the-batch/deployment-gap/. Accessed 28 Apr 2025

[67] Champaney V, Chinesta F, Cueto E. Engineering empowered by physics-based and data-driven hybrid models: a methodological overview. Int J Mater Form, 2022

[68] Crespo Márquez A (2022) The curse of dimensionality. In: Springer Ser. Reliab. Eng. https://doi.org/10.1007/978-3-030-97660-6_7

[69] Lee JA, Verleysen M (eds) (2007) Nonlinear dimensionality reduction. Springer New York, NY

[70] Li J, Ge W, Wang W, Yang N, Liu X, Wang L, He X, Wang X, Wang J, Kwauk M (2013) From multiscale modeling to Meso-Science. https://doi.org/10.1007/978-3-642-35189-1

[71] Forrester AIJ, Sóbester A, Keane AJ. Multi-fidelity optimization via surrogate modelling. Proc R Soc Lond A Math Phys Eng Sci, 2007

[72] Coşkun K, Tümer B. Learning under concept drift and non-stationary noise: introduction of the concept of persistence. Eng Appl Artif Intell, 2023

[73] Zhang J. Modern monte carlo methods for efficient uncertainty quantification and propagation: a survey. WIREs Comput Stat, 2021

[74] Gama J, Zliobaite I, Bifet A, Pechenizkiy M, Bouchachia A. A survey on concept drift adaptation. ACM Comput Surv, 2014

[75] Bayram F, Ahmed BS, Kassler A. From concept drift to model degradation: an overview on performance-aware drift detectors. Knowl-Based Syst, 2022

[76] Snyder J, Menard A, Spare N (2023) Big data = Big questions for the engineering and construction industry. https://fmicorp.com/insights/industry-insights/big-data-big-questions-for-the-engineering-and-construction-industry. Accessed 11 Jul 2023

[77] Khan N, Yaqoob I, Hashem IAT, Inayat Z, Mahmoud Ali WK, Alam M, Shiraz M, Gani A. Big data: survey, technologies, opportunities, and challenges. Sci World J, 2014

[78] Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015

[79] Delaine F, Lebental B, Rivano H. In situ calibration algorithms for environmental sensor networks: a review. IEEE Sens J, 2019, 19: 5968-5978

[80] Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proc. 25th Int Conf Mach Learn. https://doi.org/10.1145/1390156.1390294

[81] Manjunath K, Tewary S, Khatri N, Cheng K. Monitoring of machining process anomalies in diamond turning of Ti6Al4V alloy using transfer learning-based algorithms. Comput Ind Eng, 2023, 182: 109359

[82] Mattera G, Mattera R, Vespoli S, Salatiello E. Anomaly detection in manufacturing systems with temporal networks and unsupervised machine learning. Computers Ind Eng, 2025, 203: 111023

[83] Zhang P (2010) Advanced industrial control technology. https://doi.org/10.1016/C2009-0-20337-0

[84] Stuckner J, Piekenbrock M, Arnold SM, Ricks TM (2021) Optimal experimental design with fast neural network surrogate models. Comput Mater Sci. https://doi.org/10.1016/j.commatsci.2021.110747

[85] Tsamados A, Aggarwal N, Cowls J, Morley J, Roberts H, Taddeo M, Floridi L. The ethics of algorithms: key problems and solutions. AI Soc, 2022

[86] Zhao H, Gordon GJ (2022) Inherent tradeoffs in learning fair representations. J Mach Learn Res. 23(57):1–26. arXiv:1906.08386. https://doi.org/10.48550/arXiv.1906.08386

[87] Zhu K, Weyant JP. Strategic decisions of new technology adoption under asymmetric information: a game-theoretic model. Decis Sci, 2003

[88] Simpia LJ (2024) Machine learning adoption: the interplay of cost, time, and systems integration. http://www.theseus.fi/handle/10024/858584. Accessed 19 May 2025

[89] Eckstein MP, Koehler K, Welbourne LE, Akbas E. Humans, but not deep neural networks, often miss giant targets in scenes. Curr Biol, 2017

[90] Kühl N, Goutier M, Baier L, Wolff C, Martin D. Human vs. supervised machine learning: who learns patterns faster?. Cogn Syst Res, 2022

[91] Spicer J, Sanborn AN. What does the mind learn? A comparison of human and machine learning representations. Curr Opin Neurobiol, 2019

[92] Rajput D, Wang WJ, Chen CC. Evaluation of a decided sample size in machine learning applications. BMC Bioinformatics, 2023

[93] Jain AK, Chandrasekaran B (1982) 39 dimensionality and sample size considerations in pattern recognition practice. Handbook of Statistics. 2:835–855. https://doi.org/10.1016/S0169-7161(82)02042-2

[94] Raudys SJ, Jain AK. Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Trans Pattern Anal Mach Intell, 1991, 13: 252-264

[95] Baum EB, Haussler D. What size net gives valid generalization?. Neural Comput, 1989, 1: 151-160

[96] Widrow B (1987) Adaline and madaline. In: IEEE 1st Int. Conf. Neural Networks. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C23&q=Widrow%2C+B.+1987.+ADALINE+and+MADALINE+-+1963%2C+Plenary+Speech%2C+Vol+I%3A+Proc.+IEEE+Zst+Int.+Conf.+on+Neural+Networks%2C+San+Diego%2C+CA%2C+143-158.+&btnG=. Accessed 18 May 2025

[97] Alwosheel A, van Cranenburgh S, Chorus CG. Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis. J Choice Model, 2018

[98] Peduzzi P, Concato J, Kemper E, Holford TR, Feinstem AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol, 1996

[99] van Smeden M, Moons KG, de Groot JA, Collins GS, Altman DG, Eijkemans MJ, Reitsma JB (2018) Sample size for binary logistic prediction models: beyond events per variable criteria. 28:2455–2474. https://doi.org/10.1177/0962280218784726

[100] Riley RD, Snell KIE, Ensor J, Burke DL, Harrell FE, Moons KGM, Collins GS. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med, 2019

[101] Frank I, Todeschini R (1994) The data analysis handbook. https://books.google.com/books?hl=en&lr=&id=SXEpB0H6L3YC&oi=fnd&pg=PP1&ots=zfmIRO_XO5&sig=dSX6KJdkuav5zRNxaUdcftGSn2k. Accessed 21 Jun 2019

[102] IntHout J, Ioannidis JPA, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open, 2016

[103] Prasanna S, Kashinkunti P, Milletari F (2017) TensorRT 3: faster TensorFlow inference and Volta support. https://developer.nvidia.com/blog/tensorrt-3-faster-tensorflow-inference/. Accessed 4 Dec 2017

[104] Chen XW, Lin X. Big data deep learning: challenges and perspectives. IEEE Access, 2014

[105] Zhou L, Pan S, Wang J, Vasilakos AV. Machine learning on big data: opportunities and challenges. Neurocomputing, 2017

[106] Lones MA (2021) How to avoid machine learning pitfalls: a guide for academic researchers. http://www.macs.hw.ac.uk/. Accessed 10 Dec 2021

[107] Shi Y, Horiguchi M, Lu Y (2024) Suggestions on using machine learning models and cautions for analyzing censored time-to-event outcomes. JCO Precis Oncol. https://doi.org/10.1200/PO.24.00220/ASSET/5179A6CB-F38C-4794-A62B-794AB668439C/ASSETS/PO.24.00220.FP.PNG

[108] Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA, 2017

[109] Sculley D, Holt G, Golovin D, Davydov E, Phillips T, Ebner D, Chaudhary V, Young M, Crespo J-FF, Dennison D (2015) Hidden technical debt in machine learning systems. In: Proceedings of the 29th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, 2:2503–2511

[110] Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell, 2019

[111] Ribeiro MT, Singh S, Guestrin C (2016) “Why should i trust you?” Explaining the predictions of any classifier. In: Proc. ACM SIGKDD Int Conf Knowl Discov Data Min. https://doi.org/10.1145/2939672.2939778

[112] Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc, NY, United States, pp. 4768–4777

[113] Deho OB, Zhan C, Li J, Liu J, Liu L, Le TD. How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics?. Br J Educ Technol, 2022

[114] Turner Lee N. Detecting racial bias in algorithms and machine learning. J Inf Commun Ethics Soc, 2018, 16: 252-260

[115] Boenig-Liptsin M, Tanweer A, Edmundson A. Data science ethos lifecycle: interplay of ethical thinking and data science practice. J Stat Data Sci Educ, 2022

[116] Brundage M. Limitations and risks of machine ethics. J Exp Theor Artif Intell, 2014

[117] Jobin A, Ienca M, Vayena E. The global landscape of AI ethics guidelines. Nat Mach Intell, 2019, 1: 389-399

[118] Castano J, Martinez-Fernandez S, Franch X, Bogner J (2024) Analyzing the evolution and maintenance of ML models on hugging face. In: Proceedings - 2024 IEEE/ACM 21st International Conference on Mining Software Repositories, MSR 2024. 12:607–618. https://doi.org/10.1145/3643991.3644898

[119] Kang D, Raghavan D, Bailis P, Zaharia M (2020) Model assertions for monitoring and improving ML models. In: Proceedings of Machine Learning and Systems 2:481–496. https://doi.org/10.48550/arXiv.2003.01668

[120] Whang SE, Lee JG. Data collection and quality challenges for deep learning. Proc VLDB Endow, 2020

[121] Hua H, Li Y, Wang T, Dong N, Li W, Cao J. Edge computing with artificial intelligence: a machine learning perspective. ACM Comput Surv, 2023

[122] Zhou Y, Chen S, Wang Y, Huan W (2020) Review of research on lightweight convolutional neural networks. In: Proc. 2020 IEEE 5th Inf Technol Mechatronics Eng Conf. ITOEC 2020. https://doi.org/10.1109/ITOEC49072.2020.9141847

[123] Reuther A, Michaleas P, Jones M, Gadepally V, Samsi S, Kepner J (2020) Survey of machine learning accelerators. In: 2020 IEEE High Perform. Extrem Comput Conf HPEC 2020. https://doi.org/10.1109/HPEC43674.2020.9286149

[124] Molnar C (2022) Interpretable machine learning: a guide for making black box models explainable. Independently published, 28 Feb 2022

[125] Naser MZ (2023) Machine learning for all! Benchmarking automated, explainable, and coding-free platforms on civil and environmental engineering problems. J Infrastructure Intel Resilience. https://doi.org/10.1016/j.iintel.2023.100028

[126] Zhou ZH (2021) Machine learning. Springer Singapore. https://doi.org/10.1007/978-981-15-1967-3

[127] Spirtes P, Glymour C, Scheines R (1993) Causation, prediction, and search. In: Lecture Notes in Statistics. Springer, New York, 81. https://doi.org/10.1007/978-1-4612-2748-9

[128] Naser M. Causality and causal inference for engineers: beyond correlation, regression, prediction and artificial intelligence. WIREs Data Min Knowl Discov, 2024

RIGHTS & PERMISSIONS

The Author(s)