A digital twin-based quadruped robot system with scene perception, fast communication, and holographic interaction✩

Chen Zhu; Jianrong Bao; Zhouxiang Zhao; Zhaohui Yang; Chongwen Huang; Jiawen Kang; Hao Xu; Zhaoyang Zhang

doi:10.1016/j.dcan.2025.11.004

›› 2026, Vol. 12 ›› Issue (2) :262 -272. DOI: 10.1016/j.dcan.2025.11.004

Regular Papers

research-article

A digital twin-based quadruped robot system with scene perception, fast communication, and holographic interaction^✩

Author information +

^a School of Communication Engineering, Hangzhou Dianzi University, 310018, Hangzhou, China

^b Polytechnic Institute, Zhejiang University, 310015, Hangzhou, China

^c College of Information Science and Electronic Engineering, Zhejiang University, 310027, Hangzhou, China

^d Zhejiang Provincial Key Laboratory of Info. Proc., Commun. & Netw. (IPCAN), 310027, Hangzhou, China

^e School of Automation, Guangdong University of Technology, 510006, Guangzhou, China

^f Department of Electronic and Electrical Engineering, University College London, WC1E7JE, London, United Kingdom

^* E-mail addresses: zhuc@zju.edu.cn (C. Zhu), baojr@hdu.edu.cn (J. Bao), zhouxiangzhao@zju.edu.cn (Z. Zhao),

yang_zhaohui@zju.edu.cn (Z. Yang), chongwenhuang@zju.edu.cn (C. Huang), kavinkang@gdut.edu.cn (J. Kang), hao.xu@ucl.ac.uk (H. Xu), ning_ming@zju.edu.cn (Z. Zhang).

Chen Zhu received his B.Sc. degree from North University of China in 2010, and M.Sc. degree from Zhejiang University of Technology, Hangzhou, China, in 2013. He is currently engaged in teaching and research at the Polytechnic Institute, Zhejiang University. His main research interests include joint communication, sensing, and computation, machine learning, semantic communication, and cloud-edge collaborative computing.

Jianrong Bao received the Ph.D. degree in information and communication engineering from the Department of Electronic Engineering, Tsinghua University, Beijing, China, in 2009. He has been a Postdoctoral Researcher with Zhejiang University, Hangzhou, China, from 2011 to 2013 and Southeast University, Nanjing, China, from 2014 to 2017, and then a Visiting Scholar with Columbia University, New York, NY, USA, in 2015. He is cur-rently serving as a Professor with the School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, China. His main research interests include wireless commu-nications, information theory and coding, and wireless sensor network.

Zhouxiang Zhao received the B.Eng. degree in information engineering from Zhejiang University, Hangzhou, China, in 2023. He is currently pursuing the Ph.D. degree with College of Information Science and Electronic Engineering, Zhejiang University, under the supervision of Prof. Zhaohui Yang. His research interests include semantic communi-cations, graph theory, and joint communication and computation. He is the recipient of the best paper award at IEEE ICC 2024 Workshop.

Zhaohui Yang is currently a ZJU Young Professor with the Zhejiang Key Laboratory of Information Processing Communication and Networking, College of Information Science and Electronic Engineering, Zhejiang University. He received the Bachelor and Ph.D. de-grees from Southeast University, Nanjing, China, in 2014 and 2018, respectively. From 2018 to 2020, he was a Post-Doctoral Research Associate at the Center for Telecommuni-cations Research, Department of Informatics, King’s College London, U.K. From 2020 to 2022, he was a Research Fellow at the Department of Electronic and Electrical Engineer-ing, University College London, U.K. His research interests include joint communication, sensing, and computation, federated learning, and semantic communication. He received IEEE Communications Society Asia-Pacific Outstanding Young Researcher Award in 2024, IEEE Communications Society Leonard G. Abraham Prize Award in 2024, IEEE Computer Society Best Paper Award of IEEE Open Journal of the Computer Society in 2024, IEEE Marconi Prize Paper Award in 2023, IEEE Katherine Johnson Young Author Paper Award in 2023. He currently serves as an Associate Editor of IEEE TGCN, IEEE CL, IEEE TMLCN. He has served as a Guest Editor for several journals including IEEE Journal on Selected Areas in Communications.

Chongwen Huang received the B.Sc. degree from Nankai University in 2010, the M.Sc. degree from the University of Electronic Science and Technology of China in 2013, and the Ph.D. degree from Singapore University of Technology and Design (SUTD) in 2019. From October 2019 to September 2020, he was a Post-Doctoral Researcher with SUTD. In September 2020, he joined Zhejiang University as a tenure-track Young Professor. His main research interests are focused on holographic MIMO surface/reconfigurable intel-ligent surface, B5G/6G wireless communications, mmWave/THz communications, and deep learning technologies for wireless communications. He was a recipient of the 2021 IEEE Marconi Prize Paper Award, the 2023 IEEE Fred W. Ellersick Prize Paper Award, and the 2021 IEEE ComSoc Asia-Pacific Outstanding Young Researcher Award. He has been served as an Editor for IEEE Communications Letters, Signal Processing (Elsevier), EURASIP Journal on Wireless Communications and Networking, and Physical Communi-cation since 2021.

Jiawen Kang received the Ph.D. degree from the Guangdong University of Technology, China in 2018. He has been a postdoc at Nanyang Technological University, Singapore from 2018 to 2021. He currently is a full professor at Guangdong University of Technology, China. His research interests focus on blockchain, security and privacy protection.

Hao Xu received the B.S. degree in communication engineering from Nanjing University of Science and Technology, Nanjing, China, in 2013, and the Ph.D. degree in information and communication engineering from the National Mobile Communications Research Lab-oratory, Southeast University, Nanjing, in 2019. From 2019 to 2021, he was an Alexander von Humboldt (AvH) Post-Doctoral Research Fellow with the Faculty of Electrical Engi-neering and Computer Science, Technical University of Berlin, Germany. He is currently a Marie Skłodowska-Curie Actions (MSCA) Individual Fellow with the Department of Elec-tronic and Electrical Engineering, University College London, U.K. His research interests include information theory, mathematical optimization, MIMO systems, fluid antenna sys-tems, and physical layer security in wireless networks.

Zhaoyang Zhang received his Ph.D. degree from Zhejiang University, Hangzhou, China, in 1998, where he is currently a Qiushi Distinguished Professor. His research interests are mainly focused on the fundamental aspects of wireless communications and networking, with emphases on AI-empowered communications and networking, integrated communi-cation, sensing and computing, and field signal processing and communication theory, etc. He has co-authored more than 150 IEEE journal papers, and is a co-recipient of 2024 IEEE Leonard G. Abraham Prize and about 10 best paper awards or student travel grant of in-ternational conferences like IEEE ICC 2019, Globecom 2020, ISIT 2023, and WCNC 2024, etc. He was awarded the National Natural Science Fund for Distinguished Young Scholars by NSFC in 2017, and was a co-recipient of the First Grade State-level Teaching Award for Graduate Education in 2023. Dr. Zhang served as an Editor for IEEE Transactions on Wireless Communications and IEEE Transactions on Communications, etc., the Lead Guest Editor for IEEE Wireless Communications Special Issue on Sustainable Big AI Model for Wireless Networks, and as a General Chair, TPC Co-Chair or Symposium Co-Chair for WCSP 2023/2018/2013, PIMRC 2021 Workshop on Native AI Empowered Wireless Net-works, VTC-Spring 2017 Workshop on HMWC, Globecom 2014 Wireless Communications Symposium, etc. He was also a keynote speaker for IEEE Globecom 2021 Workshop on Native-AI Wireless, APCC 2018 and VTC-Fall 2017 Workshop on NOMA, etc.

Show less

History +

PDF

Abstract

This paper develops a quadruped robot virtual-real interactive control system based on digital twin technology. The system is designed to address key challenges in robotics technology, including real-time performance, low-latency control, high-precision multi-sensor data fusion, stable network transmission, data security, user-friendly interaction interface, system scalability, and maintainability. The system comprises a number of functional mod-ules, including a 3D modeling module, a positioning perception module, a virtual interaction module, a wise sensing-transmission module, and a cloud server. The 3D modeling module is responsible for constructing the virtual quadruped robot and motion space scenarios. The positioning perception module integrates LiDAR and Inertial Measurement Unit (IMU) data, utilizing Point-LIO and HDL-localization algorithms for high-precision environmental perception and positioning. The virtual interaction module provides a user-friendly control inter-face through computer software and the HoloLens headset. The wise sensing-transmission module employs WiFi and 5G links to ensure low-latency and high-bandwidth data transmission, and employs libhv and libssl asyn-chronous IO and network security cryptographic libraries to guarantee data security. The system is designed to run on the Ubuntu 20.04 platform, offering excellent scalability and maintainability. This system has broad appli-cation prospects in industrial manufacturing, construction, disaster rescue, military applications, and educational training. It enhances the performance and reliability of quadruped robot systems and lays a solid foundation for the future development of the industrial metaverse.

Keywords

Virtual-real interaction / Quadruped robot / Digital twin / Multi-sensor data fusion / Communication protocols.

Cite this article

Download citation ▾

Chen Zhu, Jianrong Bao, Zhouxiang Zhao, Zhaohui Yang, Chongwen Huang, Jiawen Kang, Hao Xu, Zhaoyang Zhang. A digital twin-based quadruped robot system with scene perception, fast communication, and holographic interaction^✩. , 2026, 12(2): 262-272 DOI:10.1016/j.dcan.2025.11.004

登录浏览全文

4963

注册一个新账户忘记密码

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work is supported in part by Zhejiang Key R&D Program un-der Grant 2023C01021, Young Elite Scientists Sponsorship Program by CAST 2023QNRC001, the Fundamental Research Funds for the Central Universities, K2023QA0AL02, Experimental Technology Research Pro-gram of Zhejiang University, SYBJS202536.

CRediT authorship contribution statement

Chen Zhu: Writing-original draft, Validation, Resources, Methodol-ogy, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization; Jianrong Bao: Writing-review & editing, Super-vision, Resources; Zhouxiang Zhao: Writing-original draft, Visualiza-tion; Zhaohui Yang: Writing-review & editing, Supervision, Resources, Project administration; Chongwen Huang: Writing-review & edit-ing, Supervision, Resources; Jiawen Kang: Writing-review & editing, Resources; Hao Xu: Writing-review & editing, Resources; Zhaoyang Zhang: Writing-review & editing, Supervision, Resources, Project ad-ministration, Funding acquisition, Conceptualization.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	W. Xu, Z. Yang, D.W.K. Ng, M. Levorato, Y.C. Eldar, M. Debbah, Edge learning for B5G networks with distributed signal processing: semantic communication, edge computing, and wireless sensing, IEEE J. Sel. Topics Signal Process. 17 (1) (2023) 9-39.

[2]	Z. Zhao, Z. Yang, M. Chen, C. Zhu, W. Xu, Z. Zhang, K. Huang, Energy-Eﬃcient probabilistic semantic communication over space-Air-Ground integrated networks, IEEE Trans. Wireless Commun. 24 (10) (2025) 8814-8829.

[3]	R.R. Murphy, Introduction to AI robotics, MIT press, 2019.

[4]	R.A. Brooks, New approaches to robotics, Science 253 (5025) (1991) 1227-1232.

[5]	M. Hägele, K. Nilsson, J.N. Pires, R. Bischoff, Industrial robotics, Springer Handbook of Robotics (2016) 1385-1422.

[6]	F.J. Folgado, D. Calderón, I. González, A.J. Calderón, Review of industry 4.0 from the perspective of automation and supervision systems: definitions, architectures and recent trends, Electronics (Basel) 13 (4) (2024).

[7]	X. Yao, N. Ma, J. Zhang, K. Wang, E. Yang, M. Faccio, Enhancing wisdom manufac-turing as industrial metaverse for industry and society 5.0, J. Intell. Manuf. 35 (1) (2024) 235-255.

[8]	L. Ren, J. Dong, L. Zhang, Y. Laili, X. Wang, Y. Qi, B.H. Li, L. Wang, L.T. Yang, M.J. Deen, Industrial metaverse for smart manufacturing: model, architecture, and applications, IEEE Trans. Cybern. (2024).

[9]	Z. Zheng, T. Li, B. Li, X. Chai, W. Song, N. Chen, Y. Zhou, Y. Lin, R. Li, Indus-trial metaverse: connotation, features, technologies, applications and challenges,in: Asian Simulation Conference, Springer, 2022, pp. 239-263.

[10]	M. Liu, S. Fang, H. Dong, C. Xu, Review of digital twin about concepts, technologies, and industrial applications, J. Manuf. Syst. 58 (2021) 346-361.

[11]	M. Singh, E. Fuenmayor, E.P. Hinchy, Y. Qiao, N. Murray, D. Devine, Digital twin: origin to future, Appl. Syst. Innovat. 4 (2) (2021) 36.

[12]	D. Mourtzis, Digital twin inception in the era of industrial metaverse, Front. Manuf. Technol. 3 (2023) 1155735.

[13]	R.Y. Zhong, X. Xu, E. Klotz, S.T. Newman,Intelligent manufacturing in the context of industry 4.0: a review, Engineering 3 (5) (2017) 616-630.

[14]	Z. Zhao, Z. Yang, C. Huang, L. Wei, Q. Yang, C. Zhong, W. Xu, Z. Zhang, A joint com-munication and computation design for distributed RISs assisted probabilistic se-mantic communication in IIot, IEEE Internet Things J. 11 (16) (2024) 26568-26579.

[15]	W. Kritzinger, M. Karner, G. Traar, J. Henjes, W. Sihn, Digital twin in manufacturing: a categorical literature review and classification, Ifac-PapersOnline 51 (11) (2018) 1016-1022.

[16]	P. Biswal, P.K. Mohanty, Development of quadruped walking robots: a review, Ain Shams Eng. J. 12 (2) (2021) 2017-2031.

[17]	D. Mourtzis, J. Angelopoulos, N. Panopoulos,The future of the human-Machine interface (HMI) in society 5.0, Future Internet 15 (5) (2023).

[18]	C. Zhu, Z. Zhao, Z. Shan, L. Yang, S. Ji, Z. Yang, Z. Zhang, Robust target detection of intelligent integrated optical camera and mmwave radar system, Digit. Signal Process. 145 (2024) 104336.

[19]	H. Durrant-Whyte, T. Bailey, Simultaneous localization and mapping: part i, IEEE Robot. Automat. Mag. 13 (2) (2006) 99-110.

[20]	T. Bailey, H. Durrant-Whyte, Simultaneous localization and mapping (SLAM): part II, IEEE Robot. Automat. Mag. 13 (3) (2006) 108-117.

[21]	D. He, W. Xu, N. Chen, F. Kong, C. Yuan, F. Zhang, Point-LIO: robust high-Bandwidth light detection and ranging inertial odometry, Adv. Intell. Syst. 5 (7) (2023) 2200459.

[22]	K. Koide, J. Miura, E. Menegatti, A portable three-dimensional LIDAR-based system for long-term and wide-area people behavior measurement, Int. J. Adv. Rob. Syst. 16 (2) (2019) 1729881419841532.

[23]	Z. Zhao, Z. Yang, M. Chen, Z. Zhang, H.V. Poor, A joint communication and com-putation design for probabilistic semantic communications, Entropy 26 (5) (Apr. 2024).

[24]	Z. Yang, M. Chen, K.-K. Wong, H.V. Poor, S. Cui,Federated learning for 6G: appli-cations, challenges, and opportunities, Engineering 8 (2022) 33-41.

[25]	F.J. Mañas-Álvarez, M. Guinaldo, R. Dormido, S. Dormido-Canto, Scalability of cyber-physical systems with real and virtual robots in ROS 2, Sensors 23 (13) (2023) 6073.

[26]	J. Guo, T. Ma, D. Weng, Synchronous mixed reality (SMR): a personalized virtual-real fusion framework with high immersion and effective interaction, J. Soc. Inf. Disp. 31 (11) (2023) 621-637.

[27]	H. Yun, M.B.G. Jun, Immersive and interactive cyber-physical system (I2CPS) and virtual reality interface for human involved robotic manufacturing, J. Manuf. Syst. 62 (2022) 234-248.

[28]	A. Bilberg, A.A. Malik, Digital twin driven human-robot collaborative assembly, CIRP Ann. 68 (1) (2019) 499-502.

[29]	K. Dröder, P. Bobka, T. Germann, F. Gabriel, F. Dietrich, A machine learning-enhanced digital twin approach for human-robot-collaboration, Procedia Cirp 76 (2018) 187-192.

[30]	L. Zhao, Z. Nie, Y. Xia, H. Li, Virtual-Physical tracking control for a car-Like mobile robot based on digital twin technology, IEEE Trans. Ind. Electron. (2024) 1-9.

[31]	D. Torielli, L. Muratore, A. Laurenzi, N. Tsagarakis, Telephysicaloperation: remote robot control based on a virtual “Marionette” type interaction interface, IEEE Rob. Autom. Lett. 7 (2) (2022) 2479-2486.

[32]	M.U. Shoukat, L. Yan, W. Liu, F. Hussain, S.A. Nawaz, A. Niaz, Digital twin-Driven virtual control technology of home-Use robot: human-Cyber-Physical system,in: 2022 17Th International Conference on Emerging Technologies (ICET), 2022, pp. 240-246.

[33]	Y. Dou, H. Wang, B. Wu, J. Cao, J. Zhang, Research on the virtual-real interaction system and interaction characteristics of a single-leg of quadruped robots based on digital twin, J. Bionic. Eng. (2024) 1-17.

[34]	C. Cruz Ulloa, J. del Cerro, A. Barrientos, Mixed-reality for quadruped-robotic guid-ance in SAR tasks, J. Comput. Des. Eng. 10 (4) (2023) 1479-1489.

[35]	Z. Zhao, Z. Yang, X. Gan, Q.-V. Pham, C. Huang, W. Xu, Z. Zhang, A joint communica-tion and computation design for semantic wireless communication with probability graph, J. Franklin Inst. 361 (13) (2024) 107055.

[36]	M.I. Ribeiro, Kalman and extended kalman filters: concept, derivation and proper-ties, Inst. Syst. Robot. 43 (46) (2004) 3736-3741.

[37]	K. He, X. Zhang, S. Ren, J. Sun,Deep residual learning for image recognition, in:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.

[38]	Z. Wu, C. Shen, A. Van Den Hengel, Wider or deeper: revisiting the resnet model for visual recognition, Pattern Recognit. 90 (2019) 119-133.

[39]	K. He, X. Zhang, S. Ren, J. Sun, Spatial pyramid pooling in deep convolutional networks for visual recognition, IEEE Trans. Pattern Anal. Mach. Intell. 37 (9) (2015) 1904-1916.

[40]	S. Park, S. Bokijonov, Y. Choi, Review of microsoft hololens applications over the past five years, Appl. Sci. 11 (16) (2021) 7259.

[41]	C. Zhu, Z. Zhao, J. Bao, Z. Yang, C. Huang, Z. Zhang,Integrated optical sensing, communication, and computation system for quadruped robots, in:2024 16Th In-ternational Conference on Wireless Communications and Signal Processing (WCSP), 2024, pp. 789-794.

[42]	Z. Zhao, Z. Yang, Y. Hu, C. Zhu, M. Shikh-Bahaei, W. Xu, Z. Zhang, K. Huang, Com-pression ratio allocation for probabilistic semantic communication with RSMA, IEEE Trans. Commun. 73 (9) (2025) 7304-7318.

[43]	M. Pharr, W. Jakob, G. Humphreys, Physically based rendering:From theory to implementation, MIT Press, 2023.

[44]	Y. Cai, W. Xu, F.Zhang, ikd-tree: An incremental kd tree for robotic applications, arXiv preprint arXiv:2102. 10808 (2021).

[45]	G.A. Einicke, L.B. White, Robust extended kalman filtering, IEEE Trans. Signal Process. 47 (9) (1999) 2596-2599.

[46]	G.A. Terejanu, et al., Extended kalman filter tutorial, University at Buffalo 27 (2008).

[47]	E.A. Wan, R. Van Der Merwe,The unscented kalman filter for nonlinear estimation, in:Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Commu-nications, and Control Symposium (Cat. No.00EX373), IEEE, 2000, pp. 153-158.

[48]	P. Biber, W. Strasser,The normal distributions transform: a new approach to laser scan matching,in:Proceedings 2003 IEEE/RSJ International Conference on Intel-ligent Robots and Systems (IROS 2003)(Cat. No.03CH37453), 3, IEEE, 2003, pp. 2743-2748.

[49]	O. Barnich, M. Van Droogenbroeck, Vibe: a universal background subtraction algo-rithm for video sequences, IEEE Trans. Image Process. 20 (6) (2010) 1709-1724.

[50]	G. Chen, L. Hong, Research on environment perception system of quadruped robots based on liDAR and vision, Drones 7 (5) (2023).

[51]

A. Naskar, P. Pothapragada, P. Vivekananda Shanmuganathan, Digital twin-Based path planning and obstacle avoidance for a quadruped robot, in: R. S. Kumar, S. Sanyal, P.M. Pathak (Recent Advances in Machines,Eds.), Mechanisms, Materials and Design, Springer Nature Singapore, Singapore, 2024, pp. 673-684.