SigmaGait: silhouette-guided motion-augmentation module for depth-based gait recognition

Abstract

Precise depth data derived from light detection and ranging (LiDAR) have been shown to be effective in improving gait recognition performance. Those depth data, however, are coarse and lack clear segmentation boundaries, which often results in increased ambiguity in interpreting precise human motion and body shape, potentially limiting the model's ability to learn discriminative gait-related features. To address this challenge, we propose a novel silhouette-guided motion-augmentation module for depth-based gait recognition, namely, SigmaGait. SigmaGait leverages the clear boundaries and homogeneous foregrounds of silhouettes to enhance the model's awareness of part-level motions and fine-grained appearance features. Furthermore, we explore a pseudo-depth generation approach that leverages accessible red-green-blue (RGB) data for scenarios where LiDAR sensors are unavailable. By employing off-the-shelf skinned multi-person linear (SMPL) models, we synthesize pseudo-depth maps from estimated human meshes, bridging the performance gap between camera and LiDAR-reliant gait recognition. We empirically evaluate our approach on real and synthetic datasets including cloth-changing benchmark for person re-identification and gait recognition (CCPG), SUSTech1K, and FreeGait. Our model achieves state-of-the-art performance on real LiDAR data, and our pseudo-depth maps improve accuracy on 2D datasets despite their synthetic origin.

Keywords

Gait recognition / Depth / Skinned multi-person linear (SMPL) / Light detection and ranging (LiDAR) / Pseudo-depth

References

[1] Ahn J , Nakashima K , Yoshino K , et al., 2025. Gait sequence upsampling using diffusion models for single LiDAR sensors. IEEE/SICE Int Symp on System Integration, p.658- 664.

[2] Chao HQ , He YW , Zhang JP , et al., 2019. GaitSet:regarding gait as a set for cross-view gait recognition. Proc 33^rd AAAI Conf on Artificial Intelligence, p.8126- 8133.

[3] Chao HQ , Wang K , He YW , et al., 2022. GaitSet:cross-view gait recognition through utilizing gait as a deep set. IEEE Trans Pattern Anal Mach Intell, 44 (7): 3467- 3478.

[4] Chu XX , Tian Z , Zhang B , et al., 2021. Proc 11^th Int Conf on Learning Representations.

[5] Cui YF , Kang YM , 2023. Multi-modal gait recognition via effective spatial-temporal feature fusion. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.17949- 17957.

[6] Dong YL , Yu CL , Ha RY , et al., 2024. HybridGait:a benchmark for spatial-temporal cloth-changing gait recognition with hybrid explorations. Proc 38^th AAAI Conf on Artificial Intelligence, p.1600- 1608.

[7] Dou HZ , Zhang PY , Su W , et al., 2023. GaitGCI:generative counterfactual intervention for gait recognition. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5578- 5588.

[8] Fan C , Peng YJ , Cao CS , et al., 2020. GaitPart:temporal part-based model for gait recognition. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.14213- 14221.

[9] Fan C , Hou SH , Huang YZ , et al., 2023a. Exploring deep models for practical gait recognition. https://arxiv.org/abs/2303.03301

[10] Fan C , Liang JH , Shen CF , et al., 2023b. OpenGait:revisiting gait recognition towards better practicality. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9707- 9716.

[11] Fan C , Ma JZ , Jin DY , et al., 2024. SkeletonGait:gait recognition using skeleton maps. Proc 38^th AAAI Conf on Artificial Intelligence, p.1662- 1669.

[12] Gao JQ , Li JQ , Shan HM , et al., 2023. Forget less, count better:a domain-incremental self-distillation learning benchmark for lifelong crowd counting. Front Inform Technol Electron Eng, 24 (2): 187- 202.

[13] Guo WX , Liang YP , Pan ZY , et al., 2024. Camera-LiDAR crossmodality gait recognition. Proc 18^th European Conf on Computer Vision, p.439- 455.

[14] Han X , Ren YM , Cong PS , et al., 2024. Gait recognition in large-scale free environment via single LiDAR. Proc 32^nd ACM Int Conf on Multimedia, p.380- 389.

[15] Heo B , Park S , Han D , et al., 2024. Rotary position embedding for vision Transformer. Proc 18^th European Conf on Computer Vision, p.289- 305.

[16] Jin DY , Fan C , Ma JZ , et al., 2025a. On denoising walking videos for gait recognition. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.12347- 12357.

[17] Jin DY , Fan C , Chen WH , et al., 2025b. Exploring more from multiple gait modalities for human identification. Proc 39^th AAAI Conf on Artificial Intelligence, p.4120- 4128.

[18] Li JQ , Zhang YZ , Shan HM , et al., 2023a. GaitCoTr:improved spatial-temporal representation for gait recognition with a hybrid convolution-Transformer framework. IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1- 5.

[19] Li JQ , Gao JQ , Zhang YZ , et al., 2023b. Motion matters:a novel motion modeling for cross-view gait feature learning. IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1- 5.

[20] Li X , Makihara Y , Xu C , et al., 2020. End-to-end model-based gait recognition. Proc 15^th Asian Conf on Computer Vision, p.3- 20.

[21] Li X , Makihara Y , Xu C , et al., 2021. End-to-end model-based gait recognition using synchronized multi-view pose constraint. Proc IEEE/CVF Int Conf on Computer Vision Workshops, p.4089- 4098.

[22] Lin BB , Zhang SL , Wang M , et al., 2022. GaitGL:learning discriminative global-local feature representations for gait recognition. https://arxiv.org/abs/2208.01380

[23] Ma K , Fu Y , Zheng DZ , et al., 2023. Dynamic aggregated network for gait recognition. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.22076- 22085.

[24] Peng YJ , Ma K , Zhang Y , et al., 2024. Learning rich features for gait recognition by integrating skeletons and silhouettes. Multim Tools Appl, 83 (3): 7273- 7294.

[25] Shen CF , Chao F , Wu W , et al., 2023. LidarGait:benchmarking 3D gait recognition with point clouds. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.1054- 1063.

[26] Shen CF , Wang R , Duan LX , et al., 2025. LidarGait++:learning local features and size awareness from LiDAR point clouds for 3D gait recognition. Proc Computer Vision and Pattern Recognition Conf, p.6627- 6636.

[27] Shin S , Kim J , Halilaj E , et al., 2024. WHAM:reconstructing worldgrounded humans with accurate 3D motion. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2070- 2080.

[28] Wang JL , Hou SH , Guo XD , et al., 2025. GaitC3I:robust crosscovariate gait recognition via causal intervention. IEEE Trans Circ Syst Video Technol, 35 (8): 8057- 8070.

[29] Wang KJ , Liu LL , Ding XN , et al., 2021. A partition approach for robust gait recognition based on gait template fusion. Front Inform Technol Electron Eng, 22 (5): 709- 719.

[30] Wang L , Liu B , Liang FF , et al., 2023. Hierarchical spatio-temporal representation learning for gait recognition. IEEE/CVF Int Conf on Computer Vision, p.19582- 19592.

[31] Wang M , Guo XD , Lin BB , et al., 2023. DyGait:exploiting dynamic representations for high-performance gait recognition. Proc IEEE/CVF Int Conf on Computer Vision, p.13378- 13387.

[32] Wang ZY , Liu J , Chen JN , et al., 2025. VM-Gait:multi-modal 3D representation based on virtual marker for gait recognition. IEEE/CVF Winter Conf on Applications of Computer Vision, p.5326- 5335.

[33] Ye DQ , Fan C , Ma JZ , et al., 2024. BigGait:learning gait representation you want by large vision models. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.200- 210.

[34] Zheng JK , Liu XC , Liu W , et al., 2022a. Gait recognition in the wild with dense 3D representations and a benchmark. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.20196- 20205.

[35] Zheng JK , Liu XC , Gu XY , et al., 2022b. Gait recognition in the wild with multi-hop temporal switch. Proc 30^th ACM Int Conf on Multimedia, p.6136- 6145.

[36] Zhu Z , Guo X , Yang T , et al., 2021. Gait recognition in the wild:a benchmark. Proc IEEE/CVF Int Conf on Computer Vision, p.14769- 14779.

[37] Zou SN , Fan C , Xiong JB , et al., 2024. Cross-covariate gait recognition:a benchmark. Proc 38^th AAAI Conf on Artificial Intelligence, p.7855- 7863.

Rights & permissions

The Authors. Published by Zhejiang University Press Co., Ltd.