Blockchain-based knowledge-aware semantic communications for remote driving image transmission^☆

Abstract

Remote driving, an emergent technology enabling remote operations of vehicles, presents a significant challenge in transmitting large volumes of image data to a central server. This requirement outpaces the capacity of traditional communication methods. To tackle this, we propose a novel framework using semantic communications, through a region of interest semantic segmentation method, to reduce the communication costs by transmitting meaningful semantic information rather than bit-wise data. To solve the knowledge base inconsistencies inherent in semantic communications, we introduce a blockchain-based edge-assisted system for managing diverse and geographically varied semantic segmentation knowledge bases. This system not only ensures the security of data through the tamper-resistant nature of blockchain but also leverages edge computing for efficient management. Additionally, the implementation of blockchain sharding handles differentiated knowledge bases for various tasks, thus boosting overall blockchain efficiency. Experimental results show a great reduction in latency by sharding and an increase in model accuracy, confirming our framework's effectiveness.

Keywords

Semantic communication / Remote driving / Semantic segmentation / Blockchain / Knowledge base management

CRediT authorship contribution statement

Yangfei Lin: Writing - original draft, Conceptualization. Tutomu Murase: Writing - review & editing. Yusheng Ji: Supervision. Wugedele Bao: Writing - review & editing. Lei Zhong: Supervision. Jie Li: Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This research was supported in part by the National Natural Science Foundation of China under Grant No. 62062031, in part by the MIC/SCOPE #JP235006102, in part by JST ASPIRE Grant Number JPMJAP2325, in part by ROIS NII Open Collaborative Research under Grant 24S0601, and in part by collaborative research with Toyota Motor Corporation, Japan.

References

[1] G. Rossolini, F. Nesti, G. D’Amico, S. Nair, A. Biondi, G. Buttazzo, On the real-world adversarial robustness of real-time semantic segmentation models for autonomous driving, IEEE Trans. Neural Netw. Learn. Syst. (2023) 1-15.

[2] F.R. Lone, H.K. Verma, K.P. Sharma, Evolution of vanets to iov: applications and challenges, Tech. Glas. Tech. J. 15 (1) (2021) 143-149.

[3] P. Zhang, W. Xu, H. Gao, K. Niu, X. Xu, X. Qin, C. Yuan, Z. Qin, H. Zhao, J. Wei, et al., Toward wisdom-evolutionary and primitive-concise 6g: a new paradigm of semantic communication networks, Engineering 8 (2022) 60-73.

[4] G. Shi, Y. Xiao, Y. Li, X. Xie, From semantic communication to semantic-aware net-working: model, architecture, and open problems, IEEE Commun. Mag. 59 (8) (2021) 44-50.

[5] D. Huang, X. Tao, F. Gao, J. Lu, Deep learning-based image semantic coding for se-mantic communications, in: 2021 IEEE Global Communications Conference (GLOBE-COM), 2021, pp. 1-6.

[6] Z. Weng, Z. Qin, Semantic communication systems for speech transmission, IEEE J. Sel. Areas Commun. 39 (8) (2021) 2434-2444.

[7] P. Jiang, C.-K. Wen, S. Jin, G.Y. Li, Wireless semantic communications for video conferencing, IEEE J. Sel. Areas Commun. 41 (1) (2022) 230-244.

[8] W. Yang, H. Du, Z.Q. Liew, W.Y.B. Lim, Z. Xiong, D. Niyato, X. Chi, X.S. Shen, C. Miao, Semantic communications for future Internet: fundamentals, applications, and challenges, IEEE Commun. Surv. Tutor. 25 (1) (2023) 213-250.

[9] Q. Pan, H. Tong, J. Lv, T. Luo, Z. Zhang, C. Yin, J. Li, Image segmentation semantic communication over Internet of vehicles, arXiv preprint, arXiv :2210.05321, 2022.

[10] W. Xu, Y. Zhang, F. Wang, Z. Qin, C. Liu, P. Zhang, Semantic communication for the Internet of vehicles: a multiuser cooperative approach, IEEE Veh. Technol. Mag. 18 (1) (2023) 100-109.

[11] W. Yang, Z.Q. Liew, W.Y.B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Cao, K.B. Letaief, Se-mantic communication meets edge intelligence, IEEE Wirel. Commun. 29 (5) (2022) 28-35.

[12] Y.E. Sagduyu, S. Ulukus, A. Yener, Task-oriented communications for nextg: end-to-end deep learning and ai security aspects, IEEE Wirel. Commun. 30 (3) (2023) 52-60.

[13] H. Du, J. Wang, D. Niyato, J. Kang, Z. Xiong, M. Guizani, D.I. Kim, Rethinking wire-less communication security in semantic Internet of things, IEEE Wirel. Commun. 30 (3) (2023) 36-43.

[14] Y. Wang, S. Guo,Transceiver cooperative learning-aided semantic communica-tions against mismatched background knowledge bases, arXiv preprint, arXiv :2301. 03133, 2023.

[15] H. Chai, S. Leng, Y. Chen, K. Zhang, A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in Internet of vehicles, IEEE Trans. Intell. Transp. Syst. 22 (7) (2020) 3975-3986.

[16] L. Peng, Z. Yang, S. Guo, X. Qiu, W. Li, P. Yu, Two-layered blockchain architecture for federated learning over mobile edge network, IEEE Netw. 36 (1) (2022) 45-51.

[17] S. Set, G. Park, Service-aware dynamic sharding approach for scalable blockchain, IEEE Trans. Serv. Comput. 16 (04) (2023) 2954-2969.

[18] Y. Lin, C. Wu, M.L. Fikri, J. Wu, J. Li, L. Zhong, Y. Ji, Blockchain-based edge-assisted knowledge base management for semantic communication in remote driv-ing, in: 2023 IEEE 31st International Conference on Network Protocols (ICNP), 2023, pp. 1-6.

[19] S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, D. Terzopoulos, Image segmentation using deep learning: a survey, IEEE Trans. Pattern Anal. Mach. Intell. 44 (7) (2021) 3523-3542.

[20] M. Toldo, A. Maracani, U. Michieli, P. Zanuttigh, Unsupervised domain adaptation in semantic segmentation: a review, Technologies 8 (2) (2020) 35.

[21] B. Diallo, J. Hu, T. Li, G.A. Khan, X. Liang, H. Wang, Auto-attention mechanism for multi-view deep embedding clustering, Pattern Recognit. (2023) 109764.

[22] P. Zhu, B. Hui, C. Zhang, D. Du, L. Wen, Q. Hu, Multi-view deep subspace clustering networks, arXiv preprint, arXiv :1908.01978, 2019.

[23] Y. Ren, J. Pu, Z. Yang, J. Xu, G. Li, X. Pu, P.S. Yu, L. He, Deep clustering: a compre-hensive survey, arXiv preprint, arXiv :2210.04142, 2022.

[24] X. Huang, M.-Y. Liu, S. Belongie, J. Kautz,Multimodal unsupervised image-to-image translation, in:Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 172-189.

[25] X. Huang, S. Belongie,Arbitrary style transfer in real-time with adaptive instance normalization, in:Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1501-1510.

[26] P.J. Rousseeuw, Silhouettes: a graphical aid to the interpretation and validation of cluster analysis, J. Comput. Appl. Math. 20 (1987) 53-65.

[27] Y. Lin, C. Wu, J. Wu, L. Zhong, X. Chen, Y. Ji, Meta-networking: beyond the Shannon limit with multi-faceted information, IEEE Netw. 37 (4) (2023) 256-264.

[28] J. Wu, C. Wu, Y. Lin, T. Yoshinaga, L. Zhong, X. Chen, Y. Ji, Semantic segmentation-based semantic communication system for image transmission, Dig. Commun. Netw. 10 (3) (2024) 519-527.

[29] E. Bourtsoulatze, D.B. Kurka, D. Gündüz, Deep joint source-channel coding for wire-less image transmission, IEEE Trans. Cogn. Commun. Netw. 5 (3) (2019) 567-579.

[30] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, The cityscapes dataset for semantic urban scene understanding, in: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 3213-3223.

[31] X. Li, K. Huang, W. Yang, S. Wang, Z. Zhang, On the convergence of fedavg on non-iid data, arXiv preprint, arXiv :1907.02189, 2019.