Align What Truly Matters: Pedestrian-Relevant Hierarchical Parsing Network for Text-Based Person Retrieval

Jiajun Su , Yifan Shi , Pudu Liu , Jianqing Zhu , Huanqiang Zeng , Liu Liu , Zhen Lei

CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (4) : 1215 -1230.

PDF (2581KB)
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (4) :1215 -1230. DOI: 10.1049/cit2.70163
ORIGINAL RESEARCH OPEN ACCESS
research-article
Align What Truly Matters: Pedestrian-Relevant Hierarchical Parsing Network for Text-Based Person Retrieval
Author information +
History +
PDF (2581KB)

Abstract

Fine-grained alignment is crucial for text-based person retrieval, which searches for relevant pedestrian images using a text query. However, background clutter and semantically vacuous words can cause interference and misalignment, hindering retrieval performance. Existing methods employ masked language modelling to randomly mask and predict words, or leverage multimodal large language models to generate diverse and pure samples. However, these methods rely on external tools and are not sufficiently stable or controllable. In this paper, we propose a pedestrian-relevant hierarchical parsing (PHP) module to extract well-aligned fine-grained visual and textual features for alignment. First, we design a coarse relevant feature mapping (CRFM) module, which uses learnable unified tokens to project both modalities into a shared low-dimensional space, enabling coarse-level semantic filtering. Next, we design an expert-driven feature parsing (EFP) module, which integrates the representational power of a mixture of experts with a modality-aware gating mechanism to uncover deep semantic associations between text and image features. Extensive experiments on three public large-scale datasets demonstrate that our approach outperforms existing state-of-the-art methods, achieving, in particular, a 5.68% improvement in mAP and a 4.81% boost in R1 accuracy on the CUHK-PEDES dataset.

Keywords

cross-modal retrieval / mixture of experts / pedestrian feature parsing

Cite this article

Download citation ▾
Jiajun Su, Yifan Shi, Pudu Liu, Jianqing Zhu, Huanqiang Zeng, Liu Liu, Zhen Lei. Align What Truly Matters: Pedestrian-Relevant Hierarchical Parsing Network for Text-Based Person Retrieval. CAAI Transactions on Intelligence Technology, 2026, 11 (4) : 1215-1230 DOI:10.1049/cit2.70163

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

Y. Wang, H. Zhang, D. Miao, and W. Pedrycz, “ Multi-Granularity Re-Ranking For Visible-Infrared Person Re-Identification,” CAAI Transactions on Intelligence Technology 8, no. 3 (2023): 770-779, https://doi.org/10.1049/cit2.12182.

[2]

D. Dai, X. Zhang, Z. Wu, H. Meng, and Z. Zhang, “ A Keypoint-Guided Feature Partition Network for Occluded Person Re-Identification,” CAAI Transactions on Intelligence Technology 10, no. 5 (2025): 1535-1547, https://doi.org/10.1049/cit2.70057.

[3]

C. Eom, G. Lee, K. Cho, H. Jung, M. Jin, and B. Ham, “ Cerberus: Attribute-Based Person Re-Identification Using Semantic IDs,” Expert Systems with Applications 259 (2025): 125320, https://doi.org/10.1016/j.eswa.2024.125320.

[4]

H. Zhang, M. Liu, Y. Li, et al., “ Attribute-Guided Collaborative Learning for Partial Person Re-Identification,” IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 12 (2023): 14144-14160, https://doi.org/10.1109/tpami.2023.3312302.

[5]

A. Radford, J. W. Kim, C. Hallacy, et al., “ Learning Transferable Visual Models From Natural Language Supervision,” International Conference on Machine Learning (2021): 8748-8763.

[6]

J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi, “ Align Before Fuse: Vision and Language Representation Learning With Momentum Distillation,” Advances in Neural Information Processing Systems 34 (2021): 9694-9705.

[7]

S. Yan, N. Dong, L. Zhang, and J. Tang, “ Clip-Driven Fine-Grained Text-Image Person Re-Identification,” IEEE Transactions on Image Processing 32 (2023): 6032-6046, https://doi.org/10.1109/tip.2023.3327924.

[8]

Y. Qin, Y. Chen, D. Peng, X. Peng, J. T. Zhou, and P. Hu, Noisy-Correspondence Learning for text-to-image Person re-identification (IEEE Conference on Computer Vision and Pattern Recognition, 2024), 27197-27206.

[9]

Y. Liu, Y. Li, X. Lan, W. Yang, Z. Liu, and Q. Liao, “ Up-Person: Unified Parameter-Efficient Transfer Learning for Text-Based Person Retrieval,” IEEE Transactions on Circuits and Systems for Video Technology 35, no. 12 (2025): 12874-12889, https://doi.org/10.1109/tcsvt.2025.3588406.

[10]

Z. Zhao, B. Liu, Y. Lu, Q. Chu, and N. Yu, “ Unifying Multi-Modal Uncertainty Modeling and Semantic Alignment for Text-To-Image Person Re-Identification,” AAAI Conference on Artificial Intelligence 38, no. 7 (2024): 7534-7542, https://doi.org/10.1609/aaai.v38i7.28585.

[11]

D. Jiang and M. Ye, “ Cross-Modal Implicit Relation Reasoning and Aligning for Text-To-Image Person Retrieval,” IEEE Conference on Computer Vision and Pattern Recognition (2023): 2787-2797.

[12]

Y. Bai, M. Cao, D. Gao, et al., “Rasa: Relation and Sensitivity Aware Representation Learning for Text-Based Person Search,” International Joint Conference on Artificial Intelligence (2023): 555-563.

[13]

J. Jiang, C. Ding, W. Tan, J. Wang, J. Tao, and X. Xu, “ Modeling Thousands of Human Annotators for Generalizable Text-To-Image Person Re-Identification,” IEEE Conference on Computer Vision and Pattern Recognition (2025): 9220-9230.

[14]

Z. Shao, X. Zhang, C. Ding, J. Wang, and J. Wang, “ Unified Pre-Training With Pseudo Texts for Text-To-Image Person Re-Identification,” IEEE International Conference on Computer Vision (2023): 11174-11184.

[15]

J. Zuo, J. Hong, F. Zhang, et al., “ Plip: Language-Image Pre-Training for Person Representation Learning,” Advances in Neural Information Processing Systems 37 (2024): 45666-45702.

[16]

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “ High-Resolution Image Synthesis With Latent Diffusion Models,” IEEE Conference on Computer Vision and Pattern Recognition (2022): 10684-10695.

[17]

A. Hurst, A. Lerer, A. P. Goucher, et al., “ Gpt-4o System Card,” arXiv preprint arXiv:2410.21276 (2024).

[18]

Wang P., Bai S., Tan S., et al., “ Qwen2-vl: Enhancing Vision-Language Model’s Perception of the World at Any Resolution,” arXiv preprint arXiv:2409.12191 (2024).

[19]

L. Yi, H. Yu, G. Wang, X. Liu, and Q. Hu, “ pFedMoE: Data-Level Personalization With Mixture Of Experts in Model-Heterogeneous Personalized Federated Learning,” IEEE Transactions on Knowledge and Data Engineering 38, no. 3 (2026): 1905-1918, https://doi.org/10.1109/tkde.2026.3656194.

[20]

B. Zoph, I. Bello, S. Kumar, et al., “ St-moe: Designing Stable and Transferable Sparse Expert Models,” arXiv preprint arXiv:2202.08906, 2022.

[21]

C. Riquelme, J. Puigcerver, B. Mustafa, et al., “ Scaling Vision With Sparse Mixture of Experts,” Advances in Neural Information Processing Systems 34 (2021): 8583-8595.

[22]

S. Li, T. Xiao, H. Li, B. Zhou, D. Yue, and X. Wang, “ Person Search With Natural Language Description,” IEEE Conference on Computer Vision and Pattern Recognition (2017): 1970-1979.

[23]

Z. Ding, C. Ding, Z. Shao, D. Tao, “ Semantically Self-Aligned Network for Text-To-Image Part-Aware Person Re-Identification,” arXiv preprint arXiv:2107.12666 (2021).

[24]

A. Zhu, Z. Wang, Y. Li, et al., “ Dssl: Deep Surroundings Person Separation Learning for Text-Based Person Retrieval,” ACM International Conference on Multimedia (2021): 209-217.

[25]

Y. Chen, G. Zhang, Y. Lu, Z. Wang, and Y. Zheng, “ Tipcb: A Simple But Effective Part-Based Convolutional Baseline for Text-Based Person Search,” Neurocomputing 494 (2022): 171-181, https://doi.org/10.1016/j.neucom.2022.04.081.

[26]

Y. Zhang and H. Lu, “ Deep Cross-Modal Projection Learning for Image-Text Matching,” European Conference on Computer Vision (2018): 686-701.

[27]

S. Li, M. Cao, and M. Zhang, “ Learning Semantic-Aligned Feature Representation for Text-Based Person Search,” IEEE International Conference on Acoustics, Speech and Signal Processing (2022): 2724-2728.

[28]

H. Wu, W. Chen, Z. Liu, T. Chen, Z. Chen, and L. Lin, “ Contrastive Transformer Learning With Proximity Data Generation for Text-Based Person Search,” IEEE Transactions on Circuits and Systems for Video Technology 34, no. 8 (2023): 7005-7016, https://doi.org/10.1109/tcsvt.2023.3329220.

[29]

L. Gao, K. Niu, B. Jiao, P. Wang, and Y. Zhang, “ Addressing Information Inequality for Text-Based Person Search via Pedestrian-Centric Visual Denoising and Bias-Aware Alignments,” IEEE Transactions on Circuits and Systems for Video Technology 33, no. 12 (2023): 7884-7899, https://doi.org/10.1109/tcsvt.2023.3273719.

[30]

X. Shu, W. Wen, H. Wu, et al., “ See Finer, See More: Implicit Modality Alignment for Text-Based Person Retrieval,” European Conference on Computer Vision Workshops (2023): 624-641.

[31]

Y. Wu, H. Wang, M. Wu, M. Cao, and M. Zhang, “ Laip: Learning Local Alignment From Image-Phrase Modeling for Text-Based Person Search,” IEEE International Conference on Multimedia and Expo (2024): 1-10.

[32]

A. Farooq, M. Awais, J. Kittler, and S. S. Khalid, “ Axm-Net: Implicit Cross-Modal Feature Alignment for Person Re-Identification,” AAAI Conference on Artificial Intelligence 36, no. 4 (2022): 4477-4485, https://doi.org/10.1609/aaai.v36i4.20370.

[33]

J. Li, D. Li, S. Savarese, and S. Hoi, “ Blip-2: Bootstrapping Language-Image Pre-Training With Frozen Image Encoders and Large Language Models,” International Conference on Machine Learning (2023): 19730-19742.

[34]

H. Han, K. Miao, Q. Zheng, and M. Luo, “ Noisy Correspondence Learning With Meta Similarity Correction,” IEEE Conference on Computer Vision and Pattern Recognition (2023): 7517-7526.

[35]

Z. Huang, G. Niu, X. Liu, et al., “ Learning With Noisy Correspondence for Cross-Modal Matching,” Advances in Neural Information Processing Systems 34 (2021): 29406-29419.

[36]

P. Hu, Z. Huang, D. Peng, X. Wang, and X. Peng, “ Cross-Modal Retrieval With Partially Mismatched Pairs,” IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 8 (2023): 9595-9610, https://doi.org/10.1109/tpami.2023.3247939.

[37]

M. Yang, Z. Huang, P. Hu, T. Li, J. Lv, and X. Peng, “ Learning With Twin Noisy Labels for Visible-Infrared Person Re-Identification,” IEEE Conference on Computer Vision and Pattern Recognition (2022): 14308-14317.

[38]

W. Suo, M. Sun, K. Niu, et al., “ A Simple and Robust Correlation Filtering Method for Text-Based Person Search,” European Conference on Computer Vision (2022): 726-742.

[39]

S. You, T. Miao, Q. Chen, X. Li, Z. Cheng, and D. Luo, “ Kpdm: Key Phrase Dynamic Masking for Robust Text-To-Image Person Retrieval,” AAAI Conference on Artificial Intelligence 40, no. 14 (2026): 12099-12107, https://doi.org/10.1609/aaai.v40i14.38199.

[40]

X. Yuan, S. Xie, L. Hua, C. Zhang, G. Zhao, and L. Y. Wu, “ Geometry-Aware Noisy Correspondence Mitigation for Cross-Modal Text-Based Person Retrieval,” AAAI Conference on Artificial Intelligence 40, no. 15 (2026): 12268-12276, https://doi.org/10.1609/aaai.v40i15.38218.

[41]

T. Yao, Y. Wu, L. Zhang, G. Sheng, Y. Li, and Q. Tian, “ Correspondence Calibrating and Dynamic Consistency Learning for Noisy Cross-Modal Retrieval,” IEEE Transactions on Multimedia (2026).

[42]

W. Cai, J. Jiang, F. Wang, J. Tang, S. Kim, and J. Huang, “ A Survey on Mixture of Experts in Large Language Models,” IEEE Transactions on Knowledge and Data Engineering 37, no. 7 (2025): 3896-3915, https://doi.org/10.1109/TKDE.2025.3554028.

[43]

Z. Cao, Q. Xu, Z. Yang, X. Cao, and Q. Huang, “ Cakge: Context-Aware Adaptive Learning for Dynamic Knowledge Graph Embeddings,” IEEE Transactions on Pattern Analysis and Machine Intelligence (2026).

[44]

T. Chen, C. Wang, Y. Zhang, K. Xia, and P. Qian, “ Dmfusion: Degradation-Customized Mixture-Of-Experts With Adaptive Discrimination for Multi-Modal Image Fusion,” IEEE Transactions on Circuits and Systems for Video Technology (2026).

[45]

T. M. Nehdi, N. Mrabah, A. Belal, M. Pedersoli, and E. Granger, “ Low-Rank Expert Merging for Multi-Source Domain Adaptation in Person Re-Identification,” Winter Conference on Applications of Computer Vision (2026): 1809-1819.

[46]

Y. Liu, Z. Liu, X. Lan, et al., “ Domain-Aware Mixture-Of-Adapters for Text-Based Person Retrieval,” AAAI Conference on Artificial Intelligence (2025): 5703-5711, https://doi.org/10.1609/aaai.v39i6.32608.

[47]

Y. Wang, Y. Zhang, X. Ren, and Y. Deng, “ Moda: Mixture of Domain Adapters for Parameter-Efficient Generalizable Person Re-Identification,” ACM Transactions on Multimedia Computing, Communications and Applications 21, no. 5 (2025): 1-19, https://doi.org/10.1145/3712595.

[48]

A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “ An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv preprint arXiv:2010.11929 (2021).

[49]

Y. Dai, X. Li, J. Liu, Z. Tong, and L. Duan, “ Generalizable Person Re-Identification With Relevance-Aware Mixture of Experts,” IEEE Conference on Computer Vision and Pattern Recognition (2021): 16145-16154.

[50]

R. Vershynin, “ High-Dimensional Probability: An Introduction With Applications in Data Science,” Cambridge Series in Statistical and Probabilistic Mathematics (2018).

[51]

W. Hoeffding, “ Probability Inequalities for Sums of Bounded Random Variables,” Journal of the American Statistical Association 58, no. 301 (1963): 13-30, https://doi.org/10.1080/01621459.1963.10500830.

[52]

J. Chen, L. Guo, J. Sun, et al., “ Eve: Efficient Vision-Language Pre-Training With Masked Prediction and Modality-Aware Moe,” AAAI Conference on Artificial Intelligence 38, no. 2 (2024): 1110-1119, https://doi.org/10.1609/aaai.v38i2.27872.

[53]

L. Wei, S. Zhang, W. Gao, and Q. Tian, “ Person Transfer GAN to Bridge Domain Gap for Person Re-Identification,” IEEE Conference on Computer Vision and Pattern Recognition (2018): 79-88.

[54]

S. He, H. Luo, W. Jiang, X. Jiang, and H. Ding, “ Vgsg: Vision-Guided Semantic-Group Network for Text-Based Person Search,” IEEE Transactions on Image Processing 33 (2024): 163-176, https://doi.org/10.1109/tip.2023.3337653.

[55]

S. Yang, Y. Zhou, Z. Zheng, Y. Wang, L. Zhu, and Y. Wu, “ Towards Unified Text-Based Person Retrieval: A Large-Scale Multi-Attribute and Language Search Benchmark,” ACM International Conference on Multimedia (2023): 4492-4501.

[56]

W. Tan, C. Ding, J. Jiang, F. Wang, Y. Zhan, and D. Tao, “ Harnessing the Power of Mllms for Transferable Text-To-Image Person reID,” IEEE Conference on Computer Vision and Pattern Recognition (2024): 17127-17137.

[57]

N. Shazeer, A. Mirhoseini, K. Maziarz, et al., “ Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer,” International Conference on Learning Representations (2017).

PDF (2581KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉