Distributed service caching with deep reinforcement learning for sustainable edge computing in large-scale AI^✩

Abstract

Increasing reliance on large-scale AI models has led to rising demand for intelligent services. The centralized cloud computing approach has limitations in terms of data transfer efficiency and response time, and as a result many service providers have begun to deploy edge servers to cache intelligent services in order to reduce transmission delay and communication energy consumption. However, finding the optimal service caching strategy remains a significant challenge due to the stochastic nature of service requests and the bulky nature of intelligent services. To deal with this, we propose a distributed service caching scheme integrating deep reinforcement learning (DRL) with mobility prediction, which we refer to as DSDM. Specifically, we employ the D3QN (Deep Double Dueling Q-Network) framework to integrate Long Short-Term Memory (LSTM) predicted mobile device locations into the service caching replacement algorithm and adopt the distributed multi-agent approach for learning and training. Experimental results demonstrate that DSDM achieves significant performance improvements in reducing communication energy consumption compared to traditional methods across various scenarios.

Keywords

Intelligent service / Edge caching / Deep reinforcement learning / Mobility prediction

References

[1] M. Miric, N. Jia, K.G. Huang, Using supervised machine learning for large-scale classification in management research: the case for identifying artificial intelligence patents, Strateg. Manag. J. 44 (2) (2023) 491-519.

[2] H. Wang, T. Fu, Y. Du, W. Gao, K. Huang, Z. Liu, P. Chandak, S. Liu, P. Van Katwyk, A. Deac, et al., Scientific discovery in the age of artificial intelligence, Na- ture 620 (7972) (2023) 47-60.

[3] M. Salvagno, F.S. Taccone, A.G. Gerli, et al., Can artificial intelligence help for sci- entific writing?, Crit. Care 27 (1) (2023) 1-5.

[4] A. Mitra, B. Bera, A.K. Das, S.S. Jamal, I. You, Impact on blockchain-based ai/ml- enabled big data analytics for cognitive internet of things environment, Comput. Commun. 197 (2023) 173-185.

[5] M. Bullo, S. Jardak, P. Carnelli, D. Gündüz, Sustainable edge intelligence through energy-aware early exiting, in: 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP), IEEE, 2023, pp. 1-6.

[6] K. Cao, Y. Liu, G. Meng, Q. Sun, An overview on edge computing research, IEEE Access 8 (2020) 85714-85728.

[7] J. Ni, X. Bai, A review of air conditioning energy performance in data centers, Renew. Sustain. Energy Rev. 67 (2017) 625-640.

[8] W.Z. Khan, E. Ahmed, S. Hakak, I. Yaqoob, A. Ahmed, Edge computing: a survey, Future Gener. Comput. Syst. 97 (2019) 219-235.

[9] T. Jeyaseelan, P. Ekambaram, J. Subramanian, T. Shamim, A comprehensive review on the current trends, challenges and future prospects for sustainable mobility, Re- new. Sustain. Energy Rev. 157 (2022) 112073.

[10] K. Haseeb, I.U. Din, A. Almogren, I. Ahmed, M. Guizani, Intelligent and secure edge-enabled computing model for sustainable cities using green internet of things, Sustain. Cities Soc. 68 (2021) 102779.

[11] W. Li, T. Yang, F.C. Delicato, P.F. Pires, Z. Tari, S.U. Khan, A.Y. Zomaya, On enabling sustainable edge computing with renewable energy resources, IEEE Commun. Mag. 56 (5) (2018) 94-101.

[12] Y. Siriwardhana, P. Porambage, M. Liyanage, M. Ylianttila, A survey on mobile augmented reality with 5g mobile edge computing: architectures, applications, and technical aspects, IEEE Commun. Surv. Tutor. 23 (2) (2021) 1160-1192.

[13] J. Yan, S. Bi, L. Duan, Y.-J.A. Zhang, Pricing-driven service caching and task of- floading in mobile edge computing, IEEE Trans. Wirel. Commun. 20 (7) (2021) 4495-4512.

[14] J. Xu, L. Chen, P. Zhou, Joint service caching and task offloading for mobile edge computing in dense networks, 2018, pp. 207-215.

[15] H. Wang, Y. Li, X. Zhao, F. Yang, An algorithm based on Markov chain to improve edge cache hit ratio for blockchain-enabled iot, China Commun. 17 (9) (2020) 66-76.

[16] K. Poularakis, J. Llorca, A.M. Tulino, I. Taylor, L. Tassiulas, Joint service placement and request routing in multi-cell mobile edge computing networks, 2019, pp. 10-18.

[17] J. Xu, L. Chen, P. Zhou, Joint service caching and task offloading for mobile edge computing in dense networks, 2018, pp. 207-215.

[18] A. Ahmed, H. Shan, A. Huang, Modeling impact of concurrent transmissions in cluster based mobile caching networks, IEEE Trans. Veh. Technol. 69 (4) (2020) 4601-4605.

[19] X. Xu, S. Tang, L. Qi, X. Zhou, F. Dai, W. Dou, Cnn partitioning and offloading for vehicular edge networks in web3, IEEE Commun. Mag. 61 (8) (2023) 36-42.

[20] S. Chakraborty, K. Mazumdar, Sustainable task offloading decision using genetic algorithm in sensor mobile edge computing, J. King Saud Univ, Comput. Inf. Sci. 34 (4) (2022) 1552-1568.

[21] W. Zhao, X. Wang, S. Jin, W. Yue, Y. Takahashi, An energy efficient task scheduling strategy in a cloud computing system and its performance evaluation using a two- dimensional continuous time Markov chain model, Electronics 8 (7) (2019) 775.

[22] Y. Dai, K. Zhang, S. Maharjan, Y. Zhang, Edge intelligence for energy-efficient com- putation offloading and resource allocation in 5g beyond, IEEE Trans. Veh. Technol. 69 (10) (2020) 12175-12186.

[23] C. Wu, F. Wu, L. Lyu, Y. Huang, X. Xie, Communication-efficient federated learning via knowledge distillation, Nat. Commun. 13 (2032) (2022) 1-8.

[24] Y. Liao, Y. Xu, H. Xu, Z. Yao, L. Wang, C. Qiao, Accelerating federated learning with data and model parallelism in edge computing, IEEE/ACM Trans. Netw. 32 (1) (2024) 904-918.

[25] R. Samir, H. El-Hennawy, H. Elbadawy, Cluster-based multi-user multi-server caching mechanism in beyond 5G/6G mec, Sensors 23 (2) (2023) 996.

[26] Y. Liu, Q. He, D. Zheng, X. Xia, F. Chen, B. Zhang, Data caching optimization in the edge computing environment, IEEE Trans. Serv. Comput. 15 (4) (2020) 2074-2085.

[27] X. Cao, J. Zhang, H.V. Poor, An optimal auction mechanism for mobile edge caching, 2018, pp. 388-399.

[28] F.N. Nur, S. Islam, N.N. Moon, A. Karim, S. Azam, B. Shanmugam, Priority-based offloading and caching in mobile edge cloud, J. Commun. Softw. Syst. 15 (2) (2019) 193-201.

[29] S. Ghosh, D.P. Agrawal, A high performance hierarchical caching framework for mobile edge computing environments, 2021, pp. 1-6.

[30] X. Xu, H. Li, Z. Li, X. Zhou, Safe: synergic data filtering for federated learning in cloud-edge computing, IEEE Trans. Ind. Inform. 19 (2) (2023) 1655-1665.

[31] Y. Yu, X. Si, C. Hu, J. Zhang, A review of recurrent neural networks: LSTM cells and network architectures, Neural Comput. 31 (7) (2019) 1235-1270.

[32] X. Zhou, M. Bilal, R. Dou, J.J.P.C. Rodrigues, Q. Zhao, J. Dai, X. Xu, Edge computa- tion offloading with content caching in 6G-enabled IoV, IEEE Trans. Intell. Transp. Syst. 25 (3) (2024) 2733-2747.

[33] Z. Ma, N. Nuermaimaiti, H. Zhang, H. Zhou, A. Nallanathan, Deployment model and performance analysis of clustered d2d caching networks under cluster-centric caching strategy, IEEE Trans. Commun. 68 (8) (2020) 4933-4945.

[34] Q. Zheng, T. Yang, Y. Kan, X. Tan, J. Yang, X. Jiang, On the analysis of cache in- validation with lru replacement, IEEE Trans. Parallel Distrib. Syst. 33 (3) (2022) 654-666.

[35] B. Jiang, P. Nain, D. Towsley, Covert cycle stealing in a single fifo server, ACM Trans. Model. Perform. Eval. Comput. Syst. 6 (2) (2021) 1-33.