Smelting stage recognition for converter steelmaking based on the convolutional recurrent neural network

Abstract

The converter steelmaking process represents a pivotal aspect of steel metallurgical production, with the characteristics of the flame at the furnace mouth serving as an indirect indicator of the internal smelting stage. Effectively identifying and predicting the smelting stage poses a significant challenge within industrial production. Traditional image-based methodologies, which rely on a single static flame image as input, demonstrate low recognition accuracy and inadequately extract the dynamic changes in smelting stage. To address this issue, the present study introduces an innovative recognition model that preprocesses flame video sequences from the furnace mouth and then employs a convolutional recurrent neural network (CRNN) to extract spatiotemporal features and derive recognition outputs. Additionally, we adopt feature layer visualization techniques to verify the model’s effectiveness and further enhance model performance by integrating the Bayesian optimization algorithm. The results indicate that the ResNet18 with convolutional block attention module (CBAM) in the convolutional layer demonstrates superior image feature extraction capabilities, achieving an accuracy of 90.70% and an area under the curve of 98.05%. The constructed Bayesian optimization-CRNN (BO-CRNN) model exhibits a significant improvement in comprehensive performance, with an accuracy of 97.01% and an area under the curve of 99.85%. Furthermore, statistics on the model’s average recognition time, computational complexity, and parameter quantity (Average recognition time: 5.49 ms, floating-point operations per second: 18260.21 M (1 M = 1 × 10⁶), parameters: 11.58 M) demonstrate superior performance. Through extensive repeated experiments on real-world datasets, the proposed CRNN model is capable of rapidly and accurately identifying smelting stages, offering a novel approach for converter smelting endpoint control.

Keywords

intelligent steelmaking / flame state recognition / deep learning / convolutional recurrent neural network

References

[1] ZhüK, CooperO, YangSL, DongQX. An extension of the AHP dummy pivot modeling applied to the restructuring of the iron and steel industry in China. IEEE Trans. Eng. Manage., 2014, 612370.

[2] ZhangRH, YangJ. State of the art in applications of machine learning in steelmaking process modeling. Int. J. Miner. Metall. Mater., 2023, 30112055.

[3] ZengL, ZhengZ, LianXY, et al.. Intelligent optimization method for the dynamic scheduling of hot metal ladles of one-ladle technology on ironmaking and steelmaking interface in steel plants. Int. J. Miner. Metall. Mater., 2023, 3091729.

[4] XinZC, ZhangJS, PengKX, et al.. Explainable machine learning model for predicting molten steel temperature in the LF refining process. Int. J. Miner. Metall. Mater., 2024, 31122657.

[5] HanY, ZhangCJ, WangL, ZhangYC. Industrial IoT for intelligent steelmaking with converter mouth flame spectrum information processed by deep learning. IEEE Trans. Ind. Inf., 2020, 1642640.

[6] BaiZ, HuangGB, WangDW, WangH, WestoverMB. Sparse extreme learning machine for classification. IEEE Trans. Cybern., 2014, 44101858.

[7] ZhouBL, KhoslaA, LapedrizaA, OlivaA, TorralbaA. Learning deep features for discriminative localization. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 20162921. Las Vegas

[8] ZhaoXM, WuYH, SongGD, LiZY, ZhangYZ, FanY. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med. Image Anal., 2018, 4398.

[9] J.H. Zhai and D.D. Song, Optimal instance subset selection from big data using genetic algorithm and open source framework, J. Big Data, 9(2022), No. 1, art. No. 87.

[10] HaqueT, YazicigilRT, PanKJ, WrightJ, KingetPR. Theory and design of a quadrature analog-to-information converter for energy-efficient wideband spectrum sensing. IEEE Trans. Circuits Syst. I: Regul. Pap., 2015, 622527

[11] ZhaoHT, ZhangCS. An online-learning-based evolutionary many-objective algorithm. Inf. Sci., 2020, 5091.

[12] M.K. Ghalati, J.B. Zhang, G.M.A.M. El-Fallah, B. Nenchev, and H.B. Dong, Toward learning steelmaking: A review on machine learning for basic oxygen furnace process, Mater. Genome Eng. Adv., 1(2023), No. 1, art. No. e6.

[13] C.J. Zhang, Y.C. Zhang, and Y. Han, Industrial cyber-physical system driven intelligent prediction model for converter end carbon content in steelmaking plants, J. Ind. Inf. Integr., 28(2022), art. No. 100356.

[14] B. Zhao, J.X. Zhao, W. Wu, F. Zhang, and T.L. Yao, Research on prediction model of converter temperature and carbon content based on spectral feature extraction, Sci. Rep., 13(2023), No. 1, art. No. 14409.

[15] ZhouMC, ZhaoQ, ChenYR. Endpoint prediction of BOF by flame spectrum and furnace mouth image based on fuzzy support vector machine. Optik, 2019, 178575.

[16] SunK, ZhuYT. A blowing endpoint judgment method for converter steelmaking based on improved DenseNet. 2022 34th Chinese Control and Decision Conference (CCDC), 20223839. Hefei

[17] LiuH, WangB, XiongX. Basic oxygen furnace steel-making end-point prediction based on computer vision and general regression neural network. Optik, 2014, 125185241.

[18] LiuH, WuQS, WangB, XiongX. BOF steelmaking endpoint real-time recognition based on flame multi-scale color difference histogram features weighted fusion method. 2016 35th Chinese Control Conference (CCC), 20163659. Chengdu

[19] U. Chadha, S.K. Selvaraj, A. Raj, et al., AI-driven techniques for controlling the metal melting production: A review, processes, enabling technologies, solutions, and research challenges, Mater. Res. Express, 9(2022), No. 7, art. No. 072001.

[20] NiuZT, QiH, SunAT, RenYT, HeMJ, GaoBH. Efficient and robust CNN–LSTM prediction of flame temperature aided light field online tomography. Sci. China Technol. Sci., 2024, 671271.

[21] X.Y. Lu, M.M. He, Z. Wang, H.W. Hu, J. Ji, and J.P. Zhu, Prediction of flashover time in a compartment fire by CNN–LSTM based deep neural network considering wearable data collection equipment, J. Build. Eng., 97(2024), art. No. 110719.

[22] X.D. Huang, X.J. Hao, B.W. Pan, S.G. Chen, S.X. Feng, and P. Pei, Combustion field prediction and diagnosis via spatiotemporal discrete U-ConvLSTM model, IEEE Photonics J., 16(2024), No. 2, art. No. 6800610.

[23] A. Carreon, S. Barwey, and V. Raman, A generative adversarial network (GAN) approach to creating synthetic flame images from experimental data, Energy AI, 13(2023), art. No. 100238.

[24] T. Hai, M.A. Ali, J.C. Zhou, et al., Feasibility and environmental assessments of a biomass gasification-based cycle next to optimization of its performance using artificial intelligence machine learning methods, Fuel, 334(2023), art. No. 126494.

[25] ChenY, LiuJY, XiongHL. Two-stage recognition algorithm for untrimmed converter steelmaking flame video, Pattern Recognition and Computer Vision. Pattern Recognition and Computer Vision: 4th Chinese Conference, 2021268. Beijing

[26] HeKM, ZhangXY, RenSQ, SunJ. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016770. Las Vegas

[27] ShuXB, ZhangLY, SunYL, TangJH. Host–parasite: Graph LSTM-in-LSTM for group activity recognition. IEEE Trans. Neural Netw. Learn. Syst., 2021, 322663.

[28] LiuYW, PeiAX, WangF, et al.. An attention-based category-aware GRU model for the next POI recommendation. Int. J. Intell. Syst., 2021, 3673174.

[29] SongSS, ZhangSQ, DongW, LiGC, PanCY. Multisource information fusion meta-learning network with convolutional block attention module for bearing fault diagnosis under limited dataset. Struct. Health Monit., 2024, 232818.

[30] S. Woo, J. Park, J.Y. Lee, and I.S. Kweon, CBAM: Convolutional block attention module, [in] V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, eds., Computer Vision–ECCV 2018 Springer, Cham., p. 3.

[31] Y.H. Liang, Y. Lin, and Q. Lu, Forecasting gold price using a novel hybrid model with ICEEMDAN and LSTM-CNNCBAM, Expert Syst. Appl., 206(2022), art. No. 117847.

[32] SelvarajuRR, CogswellM, DasA, VedantamR, ParikhD, BatraD. Grad-CAM: Visual explanations from deep networks via gradient-based localization. 2017 IEEE International Conference on Computer Vision (ICCV), 2017618. Venice

[33] C. van Zyl, X.M. Ye, and R. Naidoo, Harnessing eXplainable artificial intelligence for feature selection in time series energy forecasting: A comparative analysis of Grad-CAM and SHAP, Appl. Energy, 353(2024), art. No. 122079.

RIGHTS & PERMISSIONS

University of Science and Technology Beijing