Dual-branch spatial-temporal decoupled fusion transformer for safety action recognition in smart grid substation

Abstract

Smart grid substation operations often take place in hazardous environments and pose significant threats to the safety of power personnel. Relying solely on manual supervision can lead to inadequate oversight. In response to the demand for technology to identify improper operations in substation work scenarios, this paper proposes a substation safety action recognition technology to avoid the misoperation and enhance the safety management. In general, this paper utilizes a dual-branch transformer network to extract spatial and temporal information from the video dataset of operational behaviors in complex substation environments. Firstly, in order to capture the spatial-temporal correlation of people’s behaviors in smart grid substation, we devise a sparse attention module and a segmented linear attention module that are embedded into spatial branch transformer and temporal branch transformer respectively. To avoid the redundancy of spatial and temporal information, we fuse the temporal and spatial features using a tensor decomposition fusion module by a decoupled manner. Experimental results indicate that our proposed method accurately detects improper operational behaviors in substation work scenarios, outperforming other existing methods in terms of detection and recognition accuracy.

Keywords

A

References

[1] KomuroN, HashiguchiT, HiraiK, et al.. Development of wireless sensor nodes to monitor working environment and human mental conditions. IT Convergence and Security: Proceedings of ICITCS 2020, October 21–23, 2020, Seoul, South Korea, 2021, Singapore. Springer. 149157[C]

[2] ChenS, TangW, JiT, et al.. Detection of safety helmet wearing based on improved faster R-CNN. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), July 19–24, 2020, Glasgow, UK, 2020, New York. IEEE. 17[C]

[3] WangS. Substation personnel safety detection network based on YOLOv4. Proceedings of the 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), March 19–21, 2021, Shanghai, China, 2021, New York. IEEE. 877881[C]

[4] LiM, ZhangY, WangX. Advanced optical sensing techniques for smart grid substations. Journal of optoelectronics·laser, 2024, 35(2): 123-130[J]

[5] ZhaoH, ChenL, LiuJ. Application of deep learning in safety monitoring systems for smart grids. Optoelectronics letters, 2023, 19(5): 345-352[J]

[6] LiuW, AnguelovD, ErhanD, et al.. SSD: single shot MultiBox detector. 14th European Conference on Computer Vision, October 11–14, 2016, Amsterdam, The Netherlands, 2016, Cham. Springer International Publishing. 2137[C]

[7] DuanK, BaiS, XieL, et al.. CenterNet: keypoint triplets for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 27–November 2, 2019, Seoul, South Korea, 2019, New York. IEEE. 65696578[C]

[8] ZhouF, ZhaoH, NieZ. Safety Helmet detection based on YOLOv5. Proceedings of the 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), April 23–25, 2021, Chengdu, China, 2021, New York. IEEE. 611[C]

[9] HeL, TodorovicS. DeSTR: object detection with split transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18–24, 2022, New Orleans, LA, USA, 2022, New York. IEEE. 93779386[C]

[10] ChenS, SunP, SongY, et al.. DiffusionDet: diffusion model for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 11–17, 2023, Paris, France, 2023, New York. IEEE. 1983019843[C]

[11] WangC Y, BochkovskiyA, LiaoH Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18–24, 2023, Vancouver, BC, Canada, 2023, New York. IEEE. 74647475[C]

[12] JiangY, ZhangL, MiaoZ, et al.. Polarformer: multi-camera 3D object detection with polar transformer. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-23), February 7–14, 2023, Washington, USA, 2023, CA. AAAI Press. 10421050[C]

[13] ZhangY, WangT, ZhangX. MotrV2: bootstrapping end-to-end multi-object tracking by pretrained object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18–24, 2023, New Orleans, LA, USA, 2023, New York. IEEE. 2205622065[C]

[14] FangY, ZhangH, YanJ, et al.. UDNet: uncertainty-aware deep network for salient object detection. Pattern recognition, 2023, 134109099[J]

[15] WuZ, AllibertG, MeriauxF, et al.. HIDANet: RGB-D salient object detection via hierarchical depth awareness. IEEE transactions on image processing, 2023, 32: 2160-2173[J]

RIGHTS & PERMISSIONS

Tianjin University of Technology