MUGI-Net: A Group-Aware Pedestrian Trajectory Prediction Model for Autonomous Vehicles from First-Person View

Abstract

With the rapid development of autonomous driving, first-person view (FPV) pedestrian trajectory prediction has emerged as a key research direction to improve transportation system safety and operational efficiency. However, current studies ignore inter-pedestrian group information and long- and short-term dependence, leading to error accumulation at medium and long temporal horizons. To address these problems, we propose an FPV pedestrian trajectory prediction model dubbed MUGI-Net (Mixture of Universals and Group Interaction Network). It adopts a group pooling mechanism to adaptively aggregate group nodes and build sparse intra- and inter-group interaction graphs to fuse group interaction information. Afterward, it employs a Mixture of Universals (MoU) structure that combines MoF (Mixture of Feature Extractors) and MoA (Mixture of Architectures) to capture short-term dynamics and long-term dependencies simultaneously. Extensive experiments on the JAAD and PIE datasets show that MUGI-Net reduces the 1.5 s prediction MSE by 5% compared with the state-of-the-art AANet, and achieves the best performance on multiple key metrics, which is beneficial for autonomous driving in mixed traffic scenarios.

Keywords

First-person view / Trajectory prediction / Group interaction / Hybrid temporal encoding

Statement of the Use of Generative AI and AI-Assisted Technologies in the Writing Process

During the preparation of this manuscript, the author(s) used Deepseek for grammer polishing. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

Acknowledgments

This work is supported by General Project of National Natural Science Foundation of China NO.62576052, General Project of Jiangsu Provincial Department of Science and Technology NO.BK20250969, Changzhou City Applied Basic Project No.CJ20240039, and Changzhou Leading Innovative Talent Introduction and Cultivation Project NO.CQ20250044.

Author Contributions

Conceptualization, R.N. and B.Y.; Methodology, R.N.; Software, M.D.; Validation, S.Y. and M.D.; Formal Analysis, S.Y.; Investigation, M.D.; Resources, B.Y.; Data Curation, M.D.; Writing—Original Draft Preparation, R.N.; Writing—Review & Editing, B.Y.; Visualization, M.D.; Supervision, B.Y.; Project Administration, B.Y.; Funding Acquisition, R.N. and B.Y.

Ethics Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The research data is based on publicly available datasets on the internet.

Funding

This research was funded by General Project of National Natural Science Foundation of China under grant number 62576052; Project of Jiangsu Provincial Department of Science and Technology under grant number BK20250969; Changzhou City Applied Basic Project under grant number KYZ24020273; Changzhou Leading Innovative Talent Introduction and Cultivation Project under grant number CQ20250044.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Golchoubian M, Ghafurian M, Dautenhahn K, Azad NL. Pedestrian Trajectory Prediction in Pedestrian-Vehicle Mixed Environments: A Systematic Review. IEEE Trans. Intell. Transp. Syst. 2023, 24, 11544-11567. DOI:10.1109/TITS.2023.3291196

[2] Chen H, Liu Y, Hu C, Zhang X. Vulnerable Road User Trajectory Prediction for Autonomous Driving Using a Data-Driven Integrated Approach. IEEE Trans. Intell. Transp. Syst. 2023, 24, 7306-7317. DOI:10.1109/TITS.2023.3254809

[3] Bai J, Fang X, Fang J, Xue J, Yuan C. Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction. In Proceedings of the 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), Macau, China, 8-12 October 2022; pp. 1586-1592.

[4] Chen W, Sang H, Wang J, Zhao Z. DSTIGCN: Deformable Spatial-Temporal Interaction Graph Convolution Network for Pedestrian Trajectory Prediction. IEEE Trans. Intell. Transp. Syst. 2025, 26, 6923-6935. DOI:10.1109/TITS.2024.3525080

[5] Yang D, Zhang H, Yurtsever E, Redmill KA, Ozguner U. Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention. IEEE Trans. Intell. Veh. 2022, 7, 221-230. DOI:10.1109/TIV.2022.3162719

[6] Chen W, Sang H, Wang J, Zhao Z. DSTCNN: Deformable Spatial-Temporal Convolutional Neural Network for Pedestrian Trajectory Prediction. Inf. Sci. 2024, 666, 120455. DOI:10.1016/j.ins.2024.120455

[7] Chen X, Zhang H, Deng F, Liang J, Yang J. Stochastic Non-Autoregressive Transformer-Based Multi-Modal Pedestrian Trajectory Prediction for Intelligent Vehicles. IEEE Trans. Intell. Transp. Syst. 2023, 25, 3561-3574. DOI:10.1109/TITS.2023.3342040

[8] Fang F, Wang X, Li Z, Qian K, Zhou B. A Unified Framework for Pedestrian Trajectory Prediction and Social-Friendly Navigation. IEEE Trans. Ind. Electron. 2023, 71, 11072-11082. DOI:10.1109/TIE.2023.3342301

[9] Sharma N, Dhiman C, Indu S. Progressive Contextual Trajectory Prediction with Adaptive Gating and Fuzzy Logic Integration. IEEE Trans. Intell. Veh. 2024, 9, 6960-6970. DOI:10.1109/TIV.2024.3391898

[10] Feng A, Han C, Gong J, Yi Y, Qiu R, Cheng Y. Multi-Scale Learnable Gabor Transform for Pedestrian Trajectory Prediction from Different Perspectives. IEEE Trans. Intell. Transp. Syst. 2024, 25, 13253-13263. DOI:10.1109/TITS.2024.3421373

[11] Huang R, Ding J, Pagnucco M, Song Y. Fully Decoupling Trajectory and Scene Encoding for Lightweight Heatmap-Oriented Trajectory Prediction. IEEE Robot. Autom. Lett. 2024, 9, 9143-9150. DOI:10.1109/LRA.2024.3426376

[12] Yao Y, Atkins E, Roberson MJ, Vasudevan R, Du X. Coupling Intent and Action for Pedestrian Crossing Behavior Prediction. arXiv 2021, arXiv:2105.04133. DOI:10.48550/arXiv.2105.04133

[13] Fu M, Zhang T, Song W, Yang Y, Wang M. Trajectory Prediction-Based Local Spatio-Temporal Navigation Map for Autonomous Driving in Dynamic Highway Environments. IEEE Trans. Intell. Transp. Syst. 2021, 23, 6418-6429. DOI:10.1109/TITS.2021.3057110

[14] Phan-Minh T, Grigore EC, Boulton FA, Beijbom O, Wolff EM. CoverNet: Multimodal Behavior Prediction Using Trajectory Sets. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13-19 June 2020; pp. 14074-14083.

[15] Dendorfer P, Elflein S, Leal-Taixe L. MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10-17 October 2021; pp. 13158-13167.

[16] Su Y, Du J, Li Y, Li X, Liang R, Hua Z, et al. Trajectory Forecasting Based on Prior-Aware Directed Graph Convolutional Neural Network. IEEE Trans. Intell. Transp. Syst. 2022, 23, 16773-16785. DOI:10.1109/TITS.2022.3142248

[17] Yao Y, Atkins E, Johnson-Roberson M, Vasudevan R, Du X. BiTraP: Bi-Directional Pedestrian Trajectory Prediction with Multi-Modal Goal Estimation. IEEE Robot. Autom. Lett. 2021, 6, 1463-1470. DOI:10.1109/LRA.2021.3056339

[18] Wang C, Wang Y, Xu M, Crandall DJ. Stepwise Goal-Driven Networks for Trajectory Prediction. IEEE Robot. Autom. Lett. 2022, 7, 2716-2723. DOI:10.1109/LRA.2022.3145090

[19] Yin Z, Liu R, Xiong Z, Yuan Z. Multimodal Transformer Networks for Pedestrian Trajectory Prediction. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), Montreal, QC, Canada, 19-26 August 2021; pp. 1259-1265.

[20] Pang Y, Zhao X, Hu J, Yan H, Liu Y. Bayesian Spatio-Temporal Graph Transformer Network (B-STAR) for Multi-Aircraft Trajectory Prediction. Knowl.-Based Syst. 2022, 249, 108998. DOI:10.1016/j.knosys.2022.108998

[21] Xu C, Mao W, Zhang W, Chen S. Remember Intentions: Retrospective-Memory-Based Trajectory Prediction. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18-24 June 2022; pp. 6488-6497.

[22] Niu R, Hu C, Yang B, Chen H, Dai Z. Egocentric Pedestrian Trajectory Prediction with Agent-Wise Motion Fusion for Internet of Vehicles. IEEE Internet Things J. 2025, 12, 37659-37669. DOI:10.1109/JIOT.2025.3583722

[23] Yang B, Fan F, Ni R, Wang H, Jafaripournimchahi A, Hu H. A Multi-Task Learning Network with a Collision-Aware Graph Transformer for Traffic-Agents Trajectory Prediction. IEEE Trans. Intell. Transp. Syst. 2024, 25, 6677-6690. DOI:10.1109/TITS.2023.3345296

[24] Yang B, Yan K, Hu C, Hu H, Yu Z, Ni R. Dynamic Subclass-Balancing Contrastive Learning for Long-Tail Pedestrian Trajectory Prediction with Progressive Refinement. IEEE Trans. Autom. Sci. Eng. 2024, 22, 8645-8658. DOI:10.1109/TASE.2024.3487255

[25] Ni R, Fang L, Hu C, Cai Y, Hu H, Yang B. Privacy-Aware Pedestrian Trajectory Prediction with Dual-Channel Destination Guidance and Multi-Factor Aggregation Federated Learning. Transp. B Transp. Dyn. 2025, 13, 2601598. DOI:10.1080/21680566.2025.2601598

[26] Yang B, He C, Wang P, Chan CY, Liu X, Chen Y. TPPO: A Novel Trajectory Predictor with Pseudo Oracle. IEEE Trans. Syst. Man Cybern. Syst. 2024, 54, 2846-2859. DOI:10.1109/TSMC.2024.3351859

[27] Yuan Y, Weng X, Ou Y, Kitani K. AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10-17 October 2021; pp. 9793-9803.

[28] Sun J, Jiang Q, Lu C. Recursive Social Behavior Graph for Trajectory Prediction. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13-19 June 2020; pp. 657-666.

[29] Liu Y, Guo H, Meng Q, Chen H. Context-Aware Heterogeneous Fusion for Pedestrian Trajectory Prediction in Pedestrian-Vehicle Interactions. IEEE Sens. J. 2025, 25, 31040-31052. DOI:10.1109/JSEN.2025.3583223

[30] Bae I, Park JH, Jeon HG. Learning Pedestrian Group Representations for Multi-Modal Trajectory Prediction. In European Conference on Computer Vision; Springer Nature Switzerland: Cham, Switzerland, 2022; pp. 270-289.

[31] Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS. A Comprehensive Survey on Graph Neural Networks. IEEE Trans. Neural Netw. Learn. Syst. 2020, 32, 4-24. DOI:10.1109/TNNLS.2020.2978386

[32] Ni R, Lu S, Hu C, Yang B. Adaptive Progressive Transformer-Based Trajectory Prediction under Fine-Grained Trajectory-Scene Interaction Constraint. IEEE Trans. Autom. Sci. Eng. 2025, 22, 24498-24509. DOI:10.1109/TASE.2025.3633415

[33] Yang B, Lu Y, Wan R, Hu H, Yang C, Ni R. Meta-IRLSOT++: A Meta-Inverse Reinforcement Learning Method for Fast Adaptation of Trajectory Prediction Networks. Expert Syst. Appl. 2024, 240, 122499. DOI:10.1016/j.eswa.2024.122499

[34] Yang B, Zhu J, Yu Z, Fan F, Liu X, Ni R. Fast Adaptation Trajectory Prediction Method Based on Online Multisource Transfer Learning. IEEE Trans. Autom. Sci. Eng. 2025, 22, 1289-1304. DOI:10.1109/TASE.2024.3362980

[35] Cangea C, Veličković P, Jovanović N, Kipf T, Liò P. Towards Sparse Hierarchical Graph Classifiers. arXiv 2018, arXiv:1811.01287. DOI:10.48550/arXiv.1811.01287

[36] Gao H, Ji S. Graph U-Nets. In International Conference on Machine Learning; PMLR: Long Beach, CA, USA, 2019; pp. 2083-2092.

[37] Rasouli A, Kotseruba I, Tsotsos JK. Are They Going to Cross? A Benchmark Dataset and Baseline for Pedestrian Crosswalk Behavior. In Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy, 22-29 October 2017; pp. 206-213.

[38] Rasouli A, Kotseruba I, Kunic T, Tsotsos J. PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October-2 November 2019; pp. 6262-6271.

[39] Liu Q, Sang H, Wang J, Chen W, Liu Y. Non-Probability Sampling Network Based on Anomaly Pedestrian Trajectory Discrimination for Pedestrian Trajectory Prediction. Image Vis. Comput. 2024, 143, 104954. DOI:10.1016/j.imavis.2024.104954

[40] Lin Y, Hu C, Zhao B, Jiang H, Shan Y, Ding T, et al. Anchor-Based Multi-Modal Transformer Network for Pedestrian Trajectory and Intention Prediction. In Proceedings of the 2023 7th CAA International Conference on Vehicular Control and Intelligence (CVCI), Changsha, China, 27-29 October 2023; pp. 1-6.

[41] Wang J, Sang H, Chen W, Zhao Z. VOSTN: Variational One-Shot Transformer Network for Pedestrian Trajectory Prediction. Phys. Scr. 2024, 99, 026002. DOI:10.1088/1402-4896/ad19b7

[42] Ghiya A, AlShami AK, Kalita J. SGNetPose+: Stepwise Goal-Driven Networks with Pose Information for Trajectory Prediction in Autonomous Driving. In Proceedings of the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), Tucson, AZ, USA, 28 February-4 March 2025; pp. 629-637.