Semantic segmentation-based semantic communication system for image transmission

Abstract

With the rapid development of artificial intelligence and the widespread use of the Internet of Things, semantic communication, as an emerging communication paradigm, has been attracting great interest. Taking image transmission as an example, from the semantic communication's view, not all pixels in the images are equally important for certain receivers. The existing semantic communication systems directly perform semantic encoding and decoding on the whole image, in which the region of interest cannot be identified. In this paper, we propose a novel semantic communication system for image transmission that can distinguish between Regions Of Interest (ROI) and Regions Of Non-Interest (RONI) based on semantic segmentation, where a semantic segmentation algorithm is used to classify each pixel of the image and distinguish ROI and RONI. The system also enables high-quality transmission of ROI with lower communication overheads by transmissions through different semantic communication networks with different bandwidth requirements. An improved metric θPSNR is proposed to evaluate the transmission accuracy of the novel semantic transmission network. Experimental results show that our proposed system achieves a significant performance improvement compared with existing approaches, namely, existing semantic communication approaches and the conventional approach without semantics.

Keywords

Semantic Communication / Semantic segmentation / Image transmission / Image compression / Deep learning

References

[1] M.S. Mahdavinejad, M. Rezvan, M. Barekatain, P. Adibi, P. Barnaghi, A.P. Sheth, Machine learning for internet of things data analysis: a survey, Digital Commun. Network. 4 (3) (2018) 161-175.

[2] Z. Qin, X. Tao, J. Lu, G.Y. Li,Semantic communications: principles and challenges, arXiv preprint arXiv:2201. 01389 (2021).

[3] Y. Zhou, L. Liu, L. Wang, N. Hui, X. Cui, J. Wu, Y. Peng, Y. Qi, C. Xing, Service-aware 6g: an intelligent and open network based on the convergence of communication, computing and caching, Digital Commun. Network. 6 (3) (2020) 253-260.

[4] C.E. Shannon, A mathematical theory of communication, Bell Syst. Techn. J. 27 (3)(1948) 379-423.

[5] H. Xie, Z. Qin, G.Y. Li, B.-H. Juang,Deep learning based semantic communications: an initial investigation, in: GLOBECOM 2020-2020 IEEE Global Communications Conference, 2020, pp. 1-6.

[6] Q. Lan, D. Wen, Z. Zhang, Q. Zeng, X. Chen, P. Popovski, K. Huang, What is semantic communication? a view on conveying meaning in the era of machine intelligence, J. Commun. Info. Network. 6 (4) (2021) 336-371.

[7] X. Luo, H.-H. Chen, Q. Guo, Semantic communications: overview, open issues, and future research directions, IEEE Wireless Commun. 29 (1) (2022).

[8] P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou, G. Cottrell, Understanding convolution for semantic segmentation, in: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Ieee, 2018, pp. 1451-1460.

[9] A. Garcia-Garcia, S. Orts-Escolano, S. Oprea, V. Villena-Martinez, J. Garcia-Rodriguez,A review on deep learning techniques applied to semantic segmentation, arXiv preprint arXiv:1704.06857 (2017).

[10] R. Carnap, Y. Bar-Hillel, An outline of a theory of semantic information, in: Research Laboratory of Electronics, Massachusetts Institute of Technology, 1952.

[11] L. Floridi, Outline of a theory of strongly semantic information, Minds Mach. 14 (2)(2004) 197-221.

[12] J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, J.A. Hendler, Towards a theory of semantic communication, in: 2011 IEEE Network Science Workshop, 2011, pp. 110-117.

[13] Z. Weng, Z. Qin, G.Y. Li,Semantic communications for speech signals, in:ICC 2021-IEEE International Conference on Communications, 2021, pp. 1-6.

[14] H. Xie, Z. Qin, G.Y. Li, Task-oriented multi-user semantic communications for vqa, IEEE Wireless Commun. Lett. 11 (3) (2021) 553-557.

[15] H. Xie, Z. Qin, A lite distributed semantic communication system for internet of things, IEEE J. Sel. Area. Commun. 39 (1) (2020) 142-153.

[16] E. Bourtsoulatze, D.B. Kurka, D. Gunduz, Deep joint source-channel coding for wireless image transmission, IEEE Trans. Cognitive Commun. Network. 5 (3) (2019) 567-579.

[17] M. Yang, H.-S. Kim,Deep joint source-channel coding for wireless image transmission with adaptive rate control, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 5193-5197.

[18] Z. Zhang, Q. Yang, S. He, M. Sun, J. Chen, Wireless transmission of images with the assistance of multi-level semantic information, in: 18th International Symposium on Wireless Communication Systems, ISWCS, 2022, pp. 1-6.

[19] X. Kang, B. Song, J. Guo, Z. Qin, F.R. Yu, Task-oriented image transmission for scene classification in unmanned aerial systems, IEEE Trans. Commun. 70 (8) (2022) 5181-5192.

[20] J. Dai, S. Wang, K. Tan, Z. Si, X. Qin, K. Niu, P. Zhang, Nonlinear transform source-channel coding for semantic communications, IEEE J. Sel. Area. Commun. 40 (8)(2022) 2300-2316.

[21] O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation,in: International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 9351, 2015, pp. 234-241.

[22] R. Barrett, M. Berry, T.F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, H. Van der Vorst, Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 1994.

[23] Y. Zhang, R. Lu, B. Cao, Q. Zhang, Cooperative jamming-based physical-layer security of cooperative cognitive radio networks: system model and enabling techniques, IET Commun. 13 (5) (2019) 539-544.

[24] X. Pang, N. Zhao, J. Tang, C. Wu, D. Niyato, K.-K. Wong, Irs-assisted secure uav transmission via joint trajectory and beamforming design, IEEE Trans. Commun. 70 (2) (2021) 1140-1152.

[25] D. Zhou, K. Niu, C. Dong, Construction of polar codes in Rayleigh fading channel, IEEE Commun. Lett. 23 (3) (2019) 402-405.

[26] H. Xia, K. Alshathri, V.B. Lawrence, Y.-D. Yao, A. Montalvo, M. Rauchwerk, R. Cupo,Cellular signal identification using convolutional neural networks: awgn and Rayleigh fading channels, in: 2019 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN), 2019, pp. 1-5.

[27] M. Everingham, L. Van Gool, C.K. Williams, J. Winn, A. Zisserman, The pascal visual object classes (voc) challenge, Int. J. Comput. Vis. 88 (2) (2010) 303-338.

[28] P.K. Diederik, B. Jimmy, Adam: a method for stochastic optimization,in: 3rd International Conference on Learning Representations ICLR, 2015. [27] M. Everingham, L. Van Gool, C.K. Williams, J. Winn, A. Zisserman, The pascal visual object classes (voc) challenge, Int. J. Comput. Vis. 88 (2) (2010) 303-338.

[29] P.K. Diederik, B. Jimmy, Adam: a method for stochastic optimization,in: 3rd International Conference on Learning Representations ICLR, 2015.