Pyramid-based anti-fisheye feature enhancement preprocessing algorithm in torpedo can electrical devices: application in steel rolling process

Abstract

The steel manufacturing industry currently urgently needs highly accurate detection algorithms for electrical connection devices to slow down the time and danger of electrical connections to torpedo cans during high-temperature operations. The fisheye effect and fuzzy features of industrial cameras seriously affect accuracy and effectiveness, hindering the widespread application of object detection algorithms in the manufacturing industry. We propose a feature enhancement preprocessing algorithm for torpedo can electrical devices based on the pyramid structure that resists fisheye effects and serves to detect and locate electrical connection devices. With the aid of this preprocessing algorithm, the detection efficiency and accuracy of state-of-the-art (SOTA) object detection models are significantly improved. Experimental validation confirms the superiority of our method over other SOTA methods. With the application of our preprocessing algorithm, the production capacity of the steel plant increased by 31.8%, and material wastage caused by transportation decreased by 10.9%.

Keywords

Anti-fisheye effect (AFE) / Feature enhancement / Machine vision / Steel smelting / Torpedo can

References

[1] ConradLF, OliverML, JackRJ, et al.. Quantification of 6-degree-of-freedom chassis whole-body vibration in mobile heavy vehicles used in the steel making industry. J Low Freq Noise Vib Active Control, 2021, 31(2): 85-104.

[2] FuT, LiP, LiuS. An imbalanced small sample slab defect recognition method based on image generation. J Manuf Process, 2024, 118: 376-388.

[3] FuT, LiuS, LiP. Digital twin-driven smelting process management method for converter steelmaking. J Intell Manuf, 2024, 36: 2749-2765.

[4] FuT, LiP, LiuS. A method for quality inspection of continuous casting billet based on infrared flaw detection and 2-D image. IEEE Trans Instrum Meas, 2025, 74: 1-14.

[5] GaoX, ZhangR, YouZ, et al.. Use of hydrogen–rich gas in blast furnace ironmaking of V–bearing titanomagnetite: mass and energy balance calculations. Materials, 2022, 15176078.

[6] SemenovYS, Shumel’chikEL, GorupakhaVV, et al.. Monitoring blast furnace lining condition during five years of operation. Metallurgist, 2017, 61(34): 291-297.

[7] NiJ, ShenK, ChenY, et al.. An improved SSD-like deep network-based object detection method for indoor scenes. IEEE Trans Instrum Meas, 2023, 72: 1-15.

[8] GuoJ, ChenH, LiuB, et al.. A system and method for person identification and positioning incorporating object edge detection and scale-invariant feature transformation. Measure: J Int Measure Confeder, 2023, 223113759.

[9] ZhaiD, ZhangX, LiX, et al.. Object detection methods on compressed domain videos: an overview, comparative analysis, and new directions. Measure J Int Measure Confeder, 2023, 207112371.

[10] SunY, SongK, ZhouT, et al.. A shared method of metal object detection and living object detection based on the quality factor of detection coils for electric vehicle wireless charging. IEEE Trans Instrum Meas, 2023, 72: 1-17.

[11] ZouY, LiuC. A light-weight object detection method based on knowledge distillation and model pruning for seam tracking system. Measure J Int Measure Confeder, 2023, 220113438.

[12] XieQ, LiD, YuZ, et al.. Detecting trees in street images via deep learning with attention module. IEEE Trans Instrum Meas, 2020, 69(8): 5395-5406.

[13] DongZ, LiuY, FengY, et al.. Object detection method for high resolution remote sensing imagery based on convolutional neural networks with optimal object anchor scales. Int J Remote Sens, 2022, 43(7): 2698-2719.

[14] RenJ, HuX, ZhuL, et al.. Deep texture-aware features for camouflaged object detection. IEEE Trans Circuits Syst Video Technol, 2023, 33(3): 1157-1167.

[15] LiuZ, ZhangZ, TanY, et al.. Boosting camouflaged object detection with dual-task interactive transformer. Proceed - Int Confer Pattern Recog, 2022, 2022: 140-146.

[16] XueZ, XueN, XiaGS, et al.. Learning to calibrate straight lines for fisheye image rectification. Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), 2019, 2019: 1643-1651

[17] FanJ, ZhangJ, TaoD. SIR: self-supervised image rectification via seeing the same scene from multiple different lenses. IEEE Trans Image Process, 2020, 32: 865-877.

[18] Chao C, Hsu P, Lee H et al (2020) Self-supervised deep learning for fisheye image rectification. In: Proc 2020 IEEE international conference on acoustics, speech and signal processing (ICASSP) 2020: 2248−2252 https://doi.org/10.1109/icassp40776.2020.9054191

[19] LiT, TongG, TangH, et al.. FisheyeDet: a self-study and contour-based object detector in fisheye images. IEEE Access, 2020, 8: 71739-71751.

[20] YangCY, ChenHH. Efficient face detection in the fisheye image domain. IEEE Trans Image Process, 2021, 30: 5641-5651.

[21] WangW, WangH, ChenL, et al.. A novel soft sensor method based on stacked fusion autoencoder with feature enhancement for industrial application. Measure J Int Measure Confeder, 2023, 221113491.

[22] LiB, WangT, ZhaiY, et al.. RFIENet: RGB-thermal feature interactive enhancement network for semantic segmentation of insulator in backlight scenes. Measure J Int Measure Confeder, 2022, 205112177.

[23] NirmalaDM, SankarV. Graph based heterogeneous feature extraction for enhanced hardware trojan detection at gate-level using optimized XGBoost algorithm. Measure J Int Measure Confeder, 2023, 220113320.

[24] WangJ, YangJ, LuG, et al.. Adaptively fused attention module for the fabric defect detection. Adv Intell Syst, 2023, 522200151.

[25] QiQ, LiK, ZhengH, et al.. SGUIE-Net: semantic attention guided underwater image enhancement with multi-scale perception. IEEE Trans Image Process, 2022, 31: 6816-6830.

[26] Guo T, Wei Y, Shao H et al (2021) Research on underwater target detection method based on improved MSRCP and YOLOv3. In: Proc 2021 IEEE international conference on mechatronics and automation, ICMA 2021: 1158–1163. https://doi.org/10.1109/ICMA52036.2021.9512827

[27] HuX, LiuH, HuiY, et al.. Transformer feature enhancement network with template update for object tracking. Sensors, 2022, 22145219.

[28] WangCY, LiaoHYM, WuYH, et al.. CSPNet: a new backbone that can enhance learning capability of CNN. Proc IEEE Conf Comput Vis Pattern Recognit Workshop (CVPR Workshop), 2020, 2020: 390-391

[29] Fu H, Gong M, Wang C et al (2019) Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In: IEEE conference on computer vision and pattern recognition (CVPR) 2422−2431. https://doi.org/10.1109/CVPR.2019.00253.

[30] ZangX, LiG, GaoW. Multidirection and multiscale pyramid in transformer for video-based pedestrian retrieval. IEEE Trans Industr Inf, 2022, 18(12): 8776-8785.

[31] KoyunOC, KeserRK, AkkayaİB, et al.. Focus-and-detect: a small object detection framework for aerial images. Signal Process Image Commun, 2022, 104. 116675

[32] Qi S, Du J, Wu M et al (2022) Underwater small target detection based on deformable convolutional pyramid. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), Singapore, 2022, pp 2784−2788 https://doi.org/10.1109/ICASSP43922.2022.9746575

[33] Sahin O, Ozer S (2021) YOLODrone: improved YOLO architecture for object detection in drone images. In: 2021 44th international conference on telecommunications and signal processing (TSP), pp 361–365. https://doi.org/10.1109/TSP52935.2021.9522653

[34] LiaoX, LvS, LiD, et al.. Yolov4-mn3 for PCB surface defect detection. Appl Sci (Switzerland), 2021, 112411701.

[35] Ren S, He K, Girshick R et al (2015) Faster R-CNN: towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497

[36] WangZ, BovikAC, SheikhHR, et al.. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13(4): 600-612.

[37] BashiriFS, LaRoseER, PeissigPL, et al.. Mcindoor20000: a fully-labeled image dataset to advance indoor objects detection. Data Brief, 2018, 17: 71-75.

[38] JinH, LiL, SuH, et al.. Learn to enhance the low-light image via a multi-exposure generation and fusion method. J Vis Commun Image Represent, 2024, 100. 104127

[39] GaoF, LiL, WangJ, et al.. A lightweight feature distillation and enhancement network for super-resolution remote sensing images. Sensors, 2023, 2383906.

Funding

The Hong Kong Polytechnic University

RIGHTS & PERMISSIONS

The Author(s)