Research on key technologies of edge cache in virtual data space across WAN

Abstract

The authors of this paper have previously proposed the global virtual data space system (GVDS) to aggregate the scattered and autonomous storage resources in China’s national supercomputer grid (National Supercomputing Center in Guangzhou, National Supercomputing Center in Jinan, National Supercomputing Center in Changsha, Shanghai Supercomputing Center, and Computer Network Information Center in Chinese Academy of Sciences) into a storage system that spans the wide area network (WAN), which realizes the unified management of global storage resources in China. At present, the GVDS has been successfully deployed in the China National Grid environment. However, when accessing and sharing remote data in the WAN, the GVDS will cause redundant transmission of data and waste a lot of network bandwidth resources. In this paper, we propose an edge cache system as a supplementary system of the GVDS to improve the performance of upper-level applications accessing and sharing remote data. Specifically, we first designs the architecture of the edge cache system, and then study the key technologies of this architecture: the edge cache index mechanism based on double-layer hashing, the edge cache replacement strategy based on the GDSF algorithm, the request routing based on consistent hashing method, and the cluster member maintenance method based on the SWIM protocol. The experimental results show that the edge cache system has successfully implemented the relevant operation functions (read, write, deletion, modification, etc.) and is compatible with the POSIX interface in terms of function. Further, it can greatly reduce the amount of data transmission and increase the data access bandwidth when the accessed file is located at the edge cache system in terms of performance, i.e., its performance is close to the performance of the network file system in the local area network (LAN).

Graphical abstract

Keywords

virtual data space system / wide area network / edge cache / redundant data transmission

Jiantong Huo is a PhD candidate in the school of Computer Science and Technology, Beihang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2012. His research focuses on distributed storage system, system security and computer network

Yaowen Xu is a PhD candidate in the College of Computer Science and Technology, Zhejiang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2020. His research focuses on big storage system

Zhisheng Huo is an Assistant Professor of high performance computing center, College of Software, Beihang University, China. His research interests include big data storage and distributed storage system

Limin Xiao is a Professor in the school of Computer Science and Technology, Beihang University, China. His main research areas are computer architecture, computer system software, high performance computing, virtualization and cloud computing

Zhenxue He is currently a Full Associate Professor with Agricultural University of Hebei, China. His research interests include low power integrated circuit design and optimization, multiplevalued logic circuits and intelligent algorithm. He is a member of China Computer Federation

References

[1] Ashton K . That ‘internet of things’ thing. RFID Journal, 2009, 22( 7): 97– 114

[2] Liu L M , Wang B . Research of an end-to-end transfer mechanism for big data in CMAGrid environment. Computing Technology and Automation, 2014, 33( 1): 122– 126

[3] Wang B , Zong X , Tian H . Design and establishment of a nationwide meteorological computational grid. Journal of Applied Meteorological Science, 2010, 21( 5): 632– 640

[4] Li S , Xu L D , Zhao S . The internet of things: a survey. Information Systems Frontiers, 2015, 17( 2): 243– 259

[5] Dilley J , Maggs B , Parikh J , Prokop H , Sitaraman R , Weihl B . Globally distributed content delivery. IEEE Internet Computing, 2002, 6( 5): 50– 58

[6] Satyanarayanan M . The emergence of edge computing. Computer, 2017, 50( 1): 30– 39

[7] Su Z , Dai M , Xu Q , Li R , Fu S . Q-learning-based spectrum access for content delivery in mobile networks. IEEE Transactions on Cognitive Communications and Networking, 2020, 6( 1): 35– 47

[8] Ramaswamy L , Liu L , Iyengar A . Scalable delivery of dynamic content using a cooperative edge cache grid. IEEE Transactions on Knowledge and Data Engineering, 2007, 19( 5): 614– 630

[9] Zhao J. The case for VM-based cloudlets in mobile computing. See Docin.Com/P-1950150101 website, 2010

[10] Wilkinson S, Boshevski T, Brandoff J, Buterin V. Storj a peer-to-peer cloud storage network. See Citeseerx.ist.psu.edu/viewdoc/download;jsessionid=D732CB9C4DBDF36BFBCA7F5821A593AC?doi=10.1.1.693.785&rep=rep1&type=website, 2014

[11] Chen B , Yang C , Wang G . High-throughput opportunistic cooperative device-to-device communications with caching. IEEE Transactions on Vehicular Technology, 2017, 66( 8): 7527– 7539

[12] Tan H, Jiang H C, Han Z, Liu L, Zhao Q. Camul: online caching on multiple caches with relaying and bypassing. In: Proceedings of 2019 IEEE Conference on Computer Communications. 2019

[13] Headquarters A. Cisco wide area application services configuration guide. See Cisco.Com/C/En/Us/Support/Routers/Wide-Area-Application-Services-Waas-Software/Products-Configuration-Guides-List

[14] Berg B, Berger D S, McAllister S, Grosof I, Gunasekar S, Lu J, Uhlar M, Carrig J, Beckmann N, Harchol-Balter M, Ganger G R. The Cachelib caching engine: design and experiences at scale. In: Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation. 2020, 753– 768

[15] Cherkasova L. Improving WWW proxies performance with greedy-dual-size-frequency caching policy. See hpl.hp.com/techreports/98/HPL-98–69R1 website, 1998

[16] Karger D, Lehman E, Leighton T, Panigrahy R, Levine M, Lewin D. Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web. In: Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing. 1997, 654−663

[17] Das A, Gupta I, Motivala A. SWIM: scalable weakly-consistent infection-style process group membership protocol. In: Proceedings International Conference on Dependable Systems and Networks. 2002, 303−312

[18] Birman K . The promise, and limitations, of gossip protocols. ACM SIGOPS Operating Systems Review, 2007, 41( 5): 8– 13

Acknowledgements

The work described in this paper was supported by the National Key Research and Development Program of China (2018YFB0203901), the National Natural Science Foundation of China (Grant No. 61772053), the Hebei Youth Talents Support Project (BJ2019008), the Natural Science Foundation of Hebei Province (F2020204003).

RIGHTS & PERMISSIONS

2021 Higher Education Press 2021