Intelligent diagnosis for hot-rolled strip crown with unbalanced data using a hybrid multi-stage ensemble model

Abstract

To improve the smart manufacturing capabilities of strip hot rolling, based on digital twin (DT) and cyber-physical system (CPS), this paper proposes a data-driven approach for diagnosing hot-rolled strip crown. Since the hot rolling process features heredity, nonlinearity and strong coupling, the diagnosis of strip crown is an imbalanced problem with ill-defined decision boundaries. Conventional regression methods tend to learn more information from the majority class, which ignore the strip with unqualified crown. To address this challenge, a hybrid multi-stage ensemble model (HMSEN) is presented to classify strip crown. Initially, a novel data-resampling method that combines adaptive synthetic sampling (ADASYN) with repeated edited nearest neighbor (RENN) is proposed to assign more attention to unqualified crown. Subsequently, using the reinforced data, a multi-stage ensemble model is built to enhance the classification performance. Furthermore, the best-performing HMSEN is identified by exploring various combinations of base classifiers. The experimental results demonstrated the proposed novel resampling method outperforms comparison methods on crown dataset. Significantly, the proposed HMSEN outperforms not only the existing regression models but also the mechanism model. Therefore, HMSEN is the most robust and effective model to intelligently diagnose hot-rolled strip crown with unbalanced data.

Keywords

hot-rolled strip crown diagnosis / imbalanced multi-class classification / multi-stage ensemble modeling / data-resampling method / smart manufacturing / cyber-physical system

References

[1] PivotoD G S, de AlmeidaL F F, Da Rosa RighiR, et al. . Cyber-physical systems architectures for industrial Internet of Things applications in Industry 4.0: A literature review. Journal of Manufacturing Systems, 2021, 58: 176-192 J]

[2] KaratasM, EriskinL, DeveciM, et al. . Big data for healthcare industry 4.0: Applications, challenges and future perspectives. Expert Systems with Applications, 2022, 200: 116912 J]

[3] ZhouX-k, XuX-s, LiangW, et al. . Intelligent small object detection for digital twin in smart manufacturing with industrial cyber-physical systems. IEEE Transactions on Industrial Informatics, 2022, 18(2): 1377-1386 J]

[4] PengG-z, ChengY-l, ZhangY-f, et al. . Industrial big data-driven mechanical performance prediction for hot-rolling steel using lower upper bound estimation method. Journal of Manufacturing Systems, 2022, 65104-114 J]

[5] ZebaG, DabićM, ČičakM, et al. . Technology mining: Artificial intelligence in manufacturing. Technological Forecasting and Social Change, 2021, 171: 120971 J]

[6] LengJ-w, WangD-w, ShenW-m, et al. . Digital twins-based smart manufacturing system design in Industry 4.0: A review. Journal of Manufacturing Systems, 2021, 60119-137 J]

[7] TaoF, QiQ-l, WangL-h, et al. . Digital twins and cyber - physical systems toward smart manufacturing and industry 4.0: Correlation and comparison. Engineering, 2019, 5(4): 653-661 J]

[8] TaoF, ChengJ-f, QiQ-l, et al. . Digital twin-driven product design, manufacturing and service with big data. The International Journal of Advanced Manufacturing Technology, 2018, 94(9): 3563-3576 J]

[9] WangX-k, YangL T, WangY-h, et al. . ADTT: A highly efficient distributed tensor-train decomposition method for IIoT big data. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1573-1582 J]

[10] GehrmannC, GunnarssonM. A digital twin based industrial automation and control system security architecture. IEEE Transactions on Industrial Informatics, 2020, 16(1): 669-680 J]

[11] MückeG, PützP, GorgelsF. Methods of describing, assessing, and influencing shape deviations in strips [M]. Flat-Rolled Steel Processes, 2009, Boca Raton, CRC Press: 287298

[12] DengJ-f, SunJ, PengW, et al. . Application of neural networks for predicting hot-rolled strip crown. Applied Soft Computing, 2019, 78(C): 119-131 J]

[13] JiY-f, SongL-b, SunJ, et al. . Application of SVM and PCA-CS algorithms for prediction of strip crown in hot strip rolling. Journal of Central South University, 2021, 28(8): 2333-2344 J]

[14] LiG-t, GongD-y, LuX, et al. . Ensemble learning based methods for crown prediction of hot-rolled strip. ISIJ International, 2021, 61(5): 1603-1613 J]

[15] WangL, HeS-l, ZhaoZ-t, et al. . Prediction of hot-rolled strip crown based on Boruta and extremely randomized trees algorithms. Journal of Iron and Steel Research International, 2023, 30(5): 1022-1031 J]

[16] KoziarskiM, KrawczykB, WoźniakM. Radialbased oversampling for noisy imbalanced data classification. Neurocomputing, 2019, 343(C): 19-33 J]

[17] TsaiC F, LinW-c, HuY-h, et al. . Under-sampling class imbalanced datasets by combining clustering analysis and instance selection. Information Sciences, 2019, 477: 47-54 J]

[18] TaoX-m, LiQ, GuoW-j, et al. . Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification. Information Sciences: An International Journal, 2019, 487(C): 31-56 J]

[19] CaoC-j, WangZhe. IMCStacking: Cost-sensitive stacking learning with feature inverse mapping for imbalanced problems. Knowledge-Based Systems, 2018, 150: 27-37 J]

[20] ArefeenM A, NimiS T, RahmanM S. Neural network-based undersampling techniques. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52(2): 1111-1120 J]

[21] WeiG-l, MuW-m, SongY, et al. . An improved and random synthetic minority oversampling technique for imbalanced data. Knowledge-Based Systems, 2022, 248: 108839 J]

[22] AroraN, KaurP D. A Bolasso based consistent feature selection enabled random forest classification algorithm: An application to credit risk assessment. Applied Soft Computing, 2020, 86105936 J]

[23] MinhasA S, SinghS. A new bearing fault diagnosis approach combining sensitive statistical features with improved multiscale permutation entropy method. Knowledge-Based Systems, 2021, 218106883 J]

[24] ChowdhuryN K, KabirM A, RahmanM M, et al. . Machine learning for detecting COVID-19 from cough sounds: An ensemble-based MCDM method. Computers in Biology and Medicine, 2022, 145105405 J]

[25] MaK, ShenQ-q, SunX-y, et al. . Rockburst prediction model using machine learning based on microseismic parameters of Qinling water conveyance tunnel. Journal of Central South University, 2023, 30(1): 289-305 J]

[26] BreimanL. Random forests. Machine Language, 2001, 45(1): 5-32[J]

[27] GeurtsP, ErnstD, WehenkelL. Extremely randomized trees. Machine Learning, 2006, 63(1): 3-42 J]

[28] ChenT-q, GuestrinC. XGBoost: A scalable tree boosting system [C]. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. August 13–17, 2016, 2016, San Francisco, California, USA, ACM: 785-794

[29] KeG-l, MengQ, FinleyT, et al. . LightGBM: A highly efficient gradient boosting decision tree [C]. Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4–9, 2017, 2017, Long Beach, California, USA, ACM: 31493157

[30] ProkhorenkovaL, GusevG, VorobevA, et al. . CatBoost: Unbiased boosting with categorical features [C]. Proceedings of the 32nd International Conference on Neural Information Processing Systems. December 3–8, 2018, 2018, Montréal, Canada, ACM: 66396649

[31] BauerE, KohaviR. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 1999, 36(1): 105-139 J]

[32] WolpertD H. Stacked generalization. Neural Networks, 1992, 5(2): 241-259 J]

[33] TSCHER A, JAHRER M. The bigchaos solution to the netflix grand prize [R]. Netflix Prize Documentation. Available from: https://www.netflixprize.com/assets/GrandPrize2009_BPC_BigChaos.pdf, 2009.

[34] ZhouZ-h, FengJi. Deep forest. National Science Review, 2019, 6(1): 74-86 J]

[35] ArikS Ö, PfisterT. TabNet: Attentive interpretable tabular learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(8): 6679-6687 J]

[36] LiuH, ZhangX-y, YangY-x, et al. . Hourly traffic flow forecasting using a new hybrid modelling method. Journal of Central South University, 2022, 29(4): 1389-1402 J]

[37] LiuH, DengD-hua. An enhanced hybrid ensemble deep learning approach for forecasting daily PM_2.5. Journal of Central South University, 2022, 29(6): 2074-2083 J]

[38] RayhanF, AhmedS, MahbubA, et al. . CUSBoost: cluster-based under-sampling with boosting for imbalanced classification [C]. 2017 2nd International Conference on Computational Systems and Information Technology for Sustainable Solution (CSITSS), 2017, Bengaluru, India, IEEE: 15

[39] ChawlaN V, LazarevicA, HallL O, et al. . SMOTEBoost: improving prediction of the minority class in boosting [M]. Knowledge Discovery in Databases: PKDD 2003, 2003, Berlin, Heidelberg, Springer Berlin Heidelberg: 107119

[40] RayhanF, AhmedS, MahbubA, et al. . MEBoost: Mixing estimators with boosting for imbalanced data classification [C]. 2017 11th International Conference on Software, Knowledge, Information Management and Applications (SKIMA), 2017, Malabe, Sri Lanka, IEEE: 16

[41] ZhaoJ-k, JinJ, ChenS, et al. . A weighted hybrid ensemble method for classifying imbalanced data. Knowledge-Based Systems, 2020, 203: 106087 J]

[42] HouW-h, WangX-k, ZhangH-y, et al. . A novel dynamic ensemble selection classifier for an imbalanced data set: An application for credit risk assessment. Knowledge-Based Systems, 2020, 208106462 J]

[43] WangN, ZhaoS-y, CuiS-z, et al. . A hybrid ensemble learning method for the identification of Gang-related arson cases. Knowledge-Based Systems, 2021, 218106875 J]

[44] DengW, LiuH-l, XuJ-j, et al. . An improved quantum-inspired differential evolution algorithm for deep belief network. IEEE Transactions on Instrumentation and Measurement, 2020, 69(10): 7319-7327 J]

[45] ZhaoH-m, LiuJ, ChenH-y, et al. . Intelligent diagnosis using continuous wavelet transform and Gauss convolutional deep belief network. IEEE Transactions on Reliability, 2023, 72(2): 692-702 J]

[46] DengJ-f, SunJ, PengW, et al. . Imbalanced multiclass classification with active learning in strip rolling process. Knowledge-Based Systems, 2022, 255109754 J]

[47] NakanishiT. Application of work roll shift mill ‘HCW-Mill’ to hot strip and plate rolling. Hitachi Review, 1985, 4153-160[J]

[48] AtakaM. Rolling technology and theory for the last 100 years: The contribution of theory to innovation in strip rolling technology. ISIJ International, 2015, 55(1): 89-102 J]

[49] DingC-y, SunJ, LiX-j, et al. . A high-precision and transparent step-wise diagnostic framework for hot-rolled strip crown. Journal of Manufacturing Systems, 2023, 71: 144-157 J]

[50] FernandezA, GarciaS, HerreraF, et al. . SMOTE for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary. Journal of Artificial Intelligence Research, 2018, 61: 863-905 J]

[51] ChawlaN V, BowyerK W, HallL O, et al. . SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 16: 321-357 J]

[52] HeH-b, BaiY, GarciaE A, et al. . ADASYN: Adaptive synthetic sampling approach for imbalanced learning [C]. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008, Hong Kong, China, IEEE: 13221328

[53] GarcíaV, SánchezJ S, MollinedaR A. On the effectiveness of preprocessing methods when dealing with different levels of class imbalance. Knowledge-Based Systems, 2012, 25(1): 13-21 J]

[54] An experiment with the edited nearest-neighbor rule [J]. IEEE Transactions on Systems, Man, and Cybernetics, 1976, SMC-6(6): 448–452. DOI: https://doi.org/10.1109/TSMC.1976.4309523.

[55] LiuZ-d, LiD-yuan. Intelligent hybrid model to classify failure modes of overstressed rock masses in deep engineering. Journal of Central South University, 2023, 30(1): 156-174 J]

[56] CortesC, VapnikV. Support-vector networks. Machine Learning, 1995, 20(3): 273-297 J]

[57] LecunY, BengioY, HintonG. Deep learning. Nature, 2015, 521: 436-444 J]

[58] LiY-j, GuoH-x, LiuX, et al. . Adapted ensemble classification algorithm based on multiple classifier system and feature selection for classifying multi-class imbalanced data. Knowledge-Based Systems, 2016, 94(C): 88-104 J]

[59] CuiS-z, YinY-q, WangD-j, et al. . A stacking-based ensemble learning method for earthquake casualty prediction. Applied Soft Computing, 2021, 101: 107038 J]

[60] FangZ-c, WangY, PengL, et al. . A comparative study of heterogeneous ensemble-learning techniques for landslide susceptibility mapping. International Journal of Geographical Information Science, 2021, 35(2): 321-347 J]

[61] RoyA, CruzR M O, SabourinR, et al. . A study on combining dynamic selection and data preprocessing for imbalance learning. Neurocomputing, 2018, 286(C): 179-192 J]

[62] GuoH-x, LiY-j, ShangJ, et al. . Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 2017, 73220-239 J]

[63] BatistaG E A P A, PratiR C, MonardM C. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 20-29 J]

[64] GazzahS, Ben AmaraN E. New oversampling approaches based on polynomial fitting for imbalanced data sets [C]. 2008 The Eighth IAPR International Workshop on Document Analysis Systems, 2008, Nara, Japan, IEEE: 677684

[65] BaruaS, IslamM M, MuraseK. ProWSyn: Proximity weighted synthetic oversampling technique for imbalanced data set learning [C]. Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2013, Berlin, Heidelberg, Springer: 317328

[66] KovácsG. An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets. Applied Soft Computing, 2019, 83105662 J]

[67] Shwartz-zivR, ArmonA. Tabular data: Deep learning is not all you need. Information Fusion, 2022, 81(C): 84-90 J]

[68] WangS-m, ZhouJ, LiC-q, et al. . Rockburst prediction in hard rock mines developing bagging and boosting tree-based ensemble techniques. Journal of Central South University, 2021, 28(2): 527-542 J]

[69] GORISHNIY Y, RUBACHEV I, KHRULKOV V, BABENKO A. Revisiting deep learning models for tabular data [C]//35th Conference on Neural Information Processing Systems (NeurIPS 2021). Sydney, Australia, 2021.

[70] BORISOV V, LEEMANN T, SESSLER K, et al. Deep neural networks and tabular data: A survey [J]. IEEE Transactions on Neural Networks and Learning Systems. 2022. DOI: https://doi.org/10.1109/TNNLS.2022.3229161.