Preamble slice orderly queue access scheme in cell-free dense communication systems

Abstract

High reliability applications in dense access scenarios have become one of the main goals of 6G environments. To solve the access collision of dense Machine Type Communication (MTC) devices in cell-free communication systems, an intelligent cooperative secure access scheme based on multi-agent reinforcement learning and federated learning is proposed, that is, the Preamble Slice Orderly Queue Access (PSOQA) scheme. In this scheme, the preamble arrangement is combined with the access control. The preamble arrangement is realized by preamble slices which is from the virtual preamble pool. The access devices learn to queue orderly by deep reinforcement learning. The orderly queue weakens the random and avoids collision. A preamble slice is assigned to an orderly access queue at each access time. The orderly queue is determined by interaction information among multiple agents. With the federated reinforcement learning framework, the PSOQA scheme is implemented to guarantee the privacy and security of agents. Finally, the access performance of PSOQA is compared with other random contention schemes in different load scenarios. Simulation results show that PSOQA can not only improve the access success rate but also guarantee low-latency tolerant performances.

Keywords

Machine type communication / Random access / Reinforcement learning / Access success probability / Federated learning

Declaration of Competing Interest

No potential conflict of interest was reported by the authors.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under grants 61771255, and in part by the Provincial and Ministerial Key Laboratory Open Project under grant 20190904, and in part by the Key Technologies R&D Program of Jiangsu (Prospective and Key Technologies for Industry) under Grants BE2022067, BE2022067-1 and BE2022067-2.

References

[1] J. Ding, M. Nemati, C. Ranaweera, et al., Iot connectivity technologies and applica-tions: a survey, IEEE Access 8 (1) (2020) 67646-67673.

[2] P. Andres-Maldonado, P. Ameigeiras, J. Prados-Garzon, et al., Narrowband iot data transmission procedures for massive machine-type communications, IEEE Netw. 31 (6) (2017) 8-15.

[3] S. Ashraf, T. Ahmed, S. Saleem, Nrsm: node redeployment shrewd mechanism for wireless sensor network, Iran J. Comput. Sci. 4 (3) (2020) 1-13.

[4] Y. Gu, Q. Cui, Q. Ye, et al., Game-theoretic optimization for machine-type commu-nications under qos guarantee, IEEE Int. Things J. 6 (1) (2019) 790-800.

[5] Z. Nurlan, T. Zhukabayeva, M. Othman, et al., Wireless sensor network as a mesh: vision and challenges, IEEE Access 10 (1) (2022) 46-67.

[6] S.K. Sharma, X. Wang, Toward massive machine type communications in ultra-dense cellular iot networks: current issues and machine learning-assisted solutions, IEEE Commun. Surv. Tutor. 22 (1) (2020) 426-471.

[7] M. Matinmikko-Blue, S. Yrjölä, P. Ahokangas, Spectrum management in the 6g era: the role of regulation and spectrum sharing, in: 2020 2nd 6G Wireless Summit (6G SUMMIT), 2020, pp. 1-5.

[8] 3GPP,Study on RAN improvements for machine-type communications: TR 37.8 68 V11.0.0, 3GPP, 2011.

[9] D. Zhang, J. Liu, W. Zhou,Acb scheme based on reinforcement learning in m2m communication, in: GLOBECOM 2020-2020 IEEE Global Communications Confer-ence, IEEE, 2020, pp. 1-6.

[10] D.T. Wiriaatmadja, K.W. Choi, Hybrid random access and data transmission protocol for machine-to-machine communications in cellular networks, IEEE Trans. Wirel. Commun. 14 (1) (2015) 33-46.

[11] Y. Sun, G. Feng, S. Qin, et al., The smart handoff policy for millimeter wave hetero-geneous cellular networks, IEEE Trans. Mob. Comput. 17 (6) (2018) 1456-1468.

[12] W. Liu, Y. Cui, F. Yang, et al., Joint optimization of preamble selection and access barring for random access in mtc with general device activities, IEEE Trans. Com-mun. 70 (6) (2022) 3930-3946.

[13] D. Pacheco-Paramo, L. Tello-Oquendo, Delay-aware dynamic access control for mmtc in wireless networks using deep reinforcement learning, Comput. Netw. 182 (3) (2020) 107493.

[14] J. Moon, Y. Lim, Access control of mtc devices using reinforcement learning ap-proach, in: 2017 International Conference on Information Networking (ICOIN), IEEE, 2017, pp. 641-643.

[15] A. Mohammed, A. Khwaja, A. Anpalagan, I. Woungang, Base station selection in m2m communication using q-learning algorithm in lte-a networks, in: 2015 IEEE 29th International Conference on Advanced Information Networking and Applica-tions, IEEE, 2015, pp. 17-22.

[16] M.V. da Silva, R.D. Souza, H. Alves, et al., A noma-based q-learning random access method for machine type communications, IEEE Wirel. Commun. Lett. 9 (10) (2020) 1720-1724.

[17] S.K. Sharma, X. Wang, Collaborative distributed q-learning for rach congestion min-imization in cellular iot networks, IEEE Commun. Lett. 23 (4) (2019) 600-603.

[18] A. El-Hameed, K. Elsayed, A q-learning approach for machine-type communication random access in lte-advanced, Telecommun. Syst. 71 (3) (2019) 397-413.

[19] G. Tsoukaneri, S. Wu, Y. Wang, Probabilistic preamble selection with reinforcement learning for massive machine type communication (mtc) devices, in: 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communi-cations (PIMRC), IEEE, 2019, pp. 1-6.

[20] J. Ding, D. Qu, M. Feng, et al., Dynamic preamble-resource partitioning for critical mtc in massive mimo systems, IEEE Int. Things J. 8 (20) (2021) 15361-15371.

[21] M. Li, Z. Li, An nb-iot access solution based on beta distribution, Chin. J. Int. Thing. 5 (3) (2021) 70-77.

[22] D. Chen, C.S. Hong, Y. Zha, et al., Fedsvrg based communication efficient scheme for federated learning in mec networks, IEEE Trans. Veh. Technol. 70 (7) (2021) 7300-7304.

[23] S. Yu, X. Chen, Z. Zhou, et al., When deep reinforcement learning meets federated learning: intelligent multitimescale resource management for multiaccess edge com-puting in 5g ultradense network, IEEE Int. Things J. 8 (4) (2021) 2238-2251.

[24] G. Zhu, Y. Wang, K. Huang, Broadband analog aggregation for low-latency federated edge learning, IEEE Trans. Wirel. Commun. 19 (1) (2020) 491-506.

[25] B. Cao, L. Zhang, Y. Li, D. Feng, W. Cao, Intelligent offloading in multi-access edge computing: a state-of-the-art review and framework, IEEE Commun. Mag. 57 (3) (2019) 56-62.

[26] L. Zhang, B. Cao, Y. Li, M. Peng, G. Feng, A multi-stage stochastic programming-based offloading policy for fog enabled iot-ehealth, IEEE J. Sel. Areas Commun. 39 (2) (2021) 411-425.

[27] M. Cao, L. Zhang, B. Cao, Toward on-device federated learning: a direct acyclic graph-based blockchain approach, IEEE Trans. Neural Netw. Learn. Syst. 34 (4) (2023) 2028-2042.

[28] S. Zhou, H. Huang, W. Chen, P. Zhou, Z. Zheng, S. Guo, Pirate: a blockchain-based secure framework of distributed machine learning in 5g networks, IEEE Netw. 34 (6) (2020) 84-91.

[29] P.Z. Sotenga, K. Djouani, A.M. Kurien, Media access control in large-scale Internet of things: a review, IEEE Access 8 (1) (2020) 55834-55859.

[30] D. Wang, W. Wang, Z. Zhang, et al., Delay-optimal random access in large-scale energy harvesting iot networks based on mean field game, China Commun. 19 (4) (2022) 121-136.

[31] Q. Gao, R. Wang, H. Deng, et al., Performance analysis of cooperative multiple access with minimum interference range for large scale wireless networks, IEEE Access 10 (1) (2022) 2154-2169.

[32] M.R. Chowdhury, S. De, Delay-aware priority access classification for mas-sive machine-type communication, IEEE Trans. Veh. Technol. 70 (12) (2021) 13238-13254.

[33] N. Jiang, Y. Deng, A. Nallanathan, J. Yuan, A decoupled learning strategy for mas-sive access optimization in cellular iot networks, IEEE J. Sel. Areas Commun. 39 (3) (2021) 668-685.

[34] O. Naparstek, K. Cohen, Deep multi-user reinforcement learning for distributed dy-namic spectrum access, IEEE Trans. Wirel. Commun. 18 (1) (2019) 310-323.

[35] S. Wang, H. Liu, P.H. Gomes, et al., Deep reinforcement learning for dynamic multi-channel access in wireless networks, IEEE Trans. Cogn. Commun. Netw. 4 (2) (2018) 257-265.

[36] R.A. Addad, D.L.C. Dutra, T. Taleb, H. Flinck, Toward using reinforcement learning for trigger selection in network slice mobility, IEEE J. Sel. Areas Commun. 39 (7) (2021) 2241-2253.

[37] N. Zhao, Y.-C. Liang, D. Niyato, et al., Deep reinforcement learning for user associa-tion and resource allocation in heterogeneous cellular networks, IEEE Trans. Wirel. Commun. 18 (11) (2019) 5141-5152.

[38] P. Valente Klaine, M. Jaber, R.D. Souza, M.A. Imran, Backhaul aware user-specific cell association using q-learning, IEEE Trans. Wirel. Commun. 18 (7) (2019) 3528-3541.