Research on performance optimization of virtual data space across WAN

Abstract

For the high-performance computing in a WAN environment, the geographical locations of national supercomputing centers are scattered and the network topology is complex, so it is difficult to form a unified view of resources. To aggregate the widely dispersed storage resources of national supercomputing centers in China, we have previously proposed a global virtual data space named GVDS in the project of “High Performance Computing Virtual Data Space”, a part of the National Key Research and Development Program of China. The GVDS enables large-scale applications of the high-performance computing to run efficiently across WAN. However, the applications running on the GVDS are often data-intensive, requiring large amounts of data from multiple supercomputing centers across WANs. In this regard, the GVDS suffers from performance bottlenecks in data migration and access across WANs. To solve the above-mentioned problem, this paper proposes a performance optimization framework of GVDS including the multitask-oriented data migration method and the request access-aware IO proxy resource allocation strategy. In a WAN environment, the framework proposed in this paper can make an efficient migration decision based on the amount of migrated data and the number of multiple data sources, guaranteeing lower average migration latency when multiple data migration tasks are running in parallel. In addition, it can ensure that the thread resource of the IO proxy node is fairly allocated among different types of requests (the IO proxy is a module of GVDS), so as to improve the application’s performance across WANs. The experimental results show that the framework can effectively reduce the average data access delay of GVDS while improving the performance of the application greatly.

Graphical abstract

Keywords

storage aggregation across WANs / large-scale applications / GVDS / data migration / allocation of IO proxy resource

Jiantong Huo is a PhD candidate in the school of Computer Science and Technology, Beihang University, China. He received MS degree of College of Computer Science from Beihang University, China in 2012. His research focuses on distributed storage system, system security and computer network

Zhisheng Huo is an assistant professor of high performance computing center, School of Computer Science and Engineering, Beihang University, China. His research interests include big data storage and distributed storage system

Limin Xiao is a professor in the school of Computer Science and Technology, Beihang University, China. His main research areas are computer architecture, computer system software, high performance computing, virtualization and cloud computing

Zhenxue He is currently a full associate professor with Agricultural University of Hebei, China. His research interests include low power integrated circuit design and optimization, multiple-valued logic circuits and intelligent algorithm. He is a member of China Computer Federation

References

[1] Xiao L M, Song Y, Qin G J, Zhou H J, Wang C B, Wei B, Wei W, Huo Z S . GVDS: a global virtual data space for wide-area high-performance computing environments. Big Data Research, 2021, 7( 2): 123–146

[2] Tatebe O, Hiraga K, Soda N . Gfarm grid file system. New Generation Computing, 2010, 28( 3): 257–275

[3] Thomson A, Abadi D J. CalvinFS: consistent WAN replication and scalable metadata management for distributed file systems. In: Proceedings of the 13th USENIX Conference on File and Storage Technologies. 2015, 1−14

[4] Wrzeszcz M, Trzepla K, Słota R, Zemek K, Lichoń T, Opioła Ł, Nikolow D, Dutka Ł, Słota R, Kitowski J. Metadata organization and management for globalization of data access with onedata. In: Proceedings of the 11th International Conference on Parallel Processing and Applied Mathematics. 2015, 312−321

[5] Rong ZENG, Xiaofeng HOU, Lu ZHANG, Chao LI, Wenli ZHENG, Minyi GUO . Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities. Frontiers of Computer Science, 2022, 16( 6): 166106

[6] Ji X, Yang B, Zhang T, Ma X, Zhu X, Wang X, El-Sayed N, Zhai J, Liu W, Xue W. Automatic, Application-Aware I/O forwarding resource allocation. In: Proceedings of the 17th USENIX Conference on File and Storage Technologies. 2019, 265−279

[7] Song Y, Xiao L, Wang L, Qin G, Wei B, Yan B, Zhang C . GCSS: a global collaborative scheduling strategy for wide-area high-performance computing. Frontiers of Computer Science, 2022, 16( 5): 165105

[8] Huo J, Xu Y, Huo Z, Xiao L, He Z . Research on key technologies of edge cache in virtual data space across WAN. Frontiers of Computer Science, 2023, 17( 1): 171102

[9] Gog I, Schwarzkopf M, Gleave A, Watson R N M, Hand S. Firmament: fast, centralized cluster scheduling at scale. In: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. 2016, 99−115

[10] Goldberg A V . An efficient implementation of a scaling minimum-cost flow algorithm. Journal of Algorithms, 1997, 22( 1): 1–29

[11] Changbo KE, Fu XIAO, Zhiqiu HUANG, Fangxiong XIAO . A user requirements-oriented privacy policy self-adaption scheme in cloud computing. Frontiers of Computer Science, 2023, 17( 2): 172203

[12] Boutin E, Ekanayake J, Lin W, Shi B, Zhou J, Qian Z, Wu M, Zhou L. Apollo: scalable and coordinated scheduling for Cloud-Scale computing. In: Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation. 2014, 285−300

[13] Delimitrou C, Sanchez D, Kozyrakis C. Tarcil: reconciling scheduling speed and quality in large shared clusters. In: Proceedings of the 6th ACM Symposium on Cloud Computing. 2015, 97−110

[14] Richa A W, Mitzenmacher M, Sitaraman R . The power of two random choices: a survey of techniques and results. Combinatorial Optimization, 2001, 9: 255–304

[15] Dean J, Ghemawat S . MapReduce: simplified data processing on large clusters. Communications of the ACM, 2008, 51( 1): 107–113

[16] Isard M, Budiu M, Yu Y, Birrell A, Fetterly D. Dryad: distributed data-parallel programs from sequential building blocks. In: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems. 2007, 59−72

[17] Zhengxiong HOU, Hong SHEN, Xingshe ZHOU, Jianhua GU,Yunlan WANG, Tianhai ZHAO . Prediction of job characteristics for intelligent resource allocation in HPC systems: a survey and future directions. Frontiers of Computer Science, 2022, 16( 5): 165107

[18] Shuai XUE, Shang ZHAO, Quan CHEN, Zhuo SONG, Shanpei CHEN, Tao MA, Yong YANG, Wenli ZHENG, Minyi GUO . Kronos: towards bus contention-aware job scheduling in warehouse scale computers. Frontiers of Computer Science, 2023, 17( 1): 171101

[19] Schwarzkopf M, Konwinski A, Abd-El-Malek M, Wilkes J. Omega: flexible, scalable schedulers for large compute clusters. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 351−364

[20] Carrión C . Kubernetes scheduling: taxonomy, ongoing issues and challenges. ACM Computing Surveys, 2023, 55( 7): 138

[21] Park G. A generalization of multiple choice balls-into-bins. In: Proceedings of the 30th Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing. 2011, 297−298

[22] Chang H S, Givan R, Chong E K P. On-line scheduling via sampling. In: Proceedings of the 5th International Conference on Artificial Intelligence Planning Systems. 2000, 62−71

[23] Dong X, Wang Y, Liao H. Scheduling mixed real-time and non-real-time applications in MapReduce environment. In: Proceedings of the 17th IEEE International Conference on Parallel and Distributed Systems. 2011, 9−16

[24] Ousterhout K, Wendell P, Zaharia M, Stoica I. Sparrow: distributed, low latency scheduling. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles. 2013, 69–84

[25] Delimitrou C, Kozyrakis C . Paragon: QoS-aware scheduling for heterogeneous datacenters. ACM SIGPLAN Notices, 2013, 48( 4): 77–88

[26] Delimitrou C, Kozyrakis C . Quasar: resource-efficient and QoS-aware cluster management. ACM SIGPLAN Notices, 2014, 49( 4): 127–144

[27] Verma A, Pedrosa L, Korupolu M, Oppenheimer D, Tune E, Wilkes J. Large-scale cluster management at Google with Borg. In: Proceedings of the 10th European Conference on Computer Systems. 2015, 18

[28] Tumanov A, Zhu T, Park J W, Kozuch M A, Harchol-Balter M, Ganger G R. TetriSched: global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters. In: Proceedings of the 11th European Conference on Computer Systems. 2016, 35

[29] Khallouli W, Huang J . Cluster resource scheduling in cloud computing: literature review and research challenges. The Journal of Supercomputing, 2022, 78( 5): 6898–6943

[30] Curino C, Difallah D E, Douglas C, Krishnan S, Ramakrishnan R, Rao S. Reservation-based scheduling: if you’re late don’t blame us! In: Proceedings of the ACM Symposium on Cloud Computing. 2014, 1−14

[31] Wang Z, Zhang G, Wang Y, Yang Q, Zhu J. Dayu: fast and low-interference data recovery in very-large storage systems. In: Proceedings of 2019 USENIX Conference on Usenix Annual Technical Conference. 2019, 993−1007

[32] Ongaro D, Rumble S M, Stutsman R, Ousterhout J, Rosenblum M. Fast crash recovery in RAMCloud. In: Proceedings of the 23rd ACM Symposium on Operating Systems Principles. 2011, 29−41

[33] Chang F, Dean J, Ghemawat S, Hsieh W C, Wallach D A, Burrows M, Chandra T, Fikes A, Gruber R E . Bigtable: a distributed storage system for structured data. ACM Transactions on Computer Systems, 2008, 26( 2): 4

[34] Chowdhury M, Zaharia M, Ma J, Jordan M I, Stoica I . Managing data transfers in computer clusters with orchestra. ACM SIGCOMM Computer Communication Review, 2011, 41( 4): 98–109

[35] He X, Yang B, Gao J, Xiao W, Chen Q, Shi S, Chen D, Liu W, Xue W, Chen Z. HadaFS: a file system bridging the local and shared burst buffer for exascale supercomputers. In: Proceedings of the 21st USENIX Conference on File and Storage Technologies. 2023, 215−230

[36] Diao Y, Hellerstein J L, Parekh S, Shaikh H, Surendra M, Tantawi A. Modeling differentiated services of multi-tier web applications. In: Proceedings of the 14th IEEE International Symposium on Modeling, Analysis, and Simulation. 2006, 314−326

[37] Lu C, Lu Y, Abdelzaher T F, Stankovic J A, Son S H . Feedback control architecture and design methodology for service delay guarantees in Web servers. IEEE Transactions on Parallel and Distributed Systems, 2006, 17( 9): 1014–1027

[38] Zhang Y, Jiang J, Xu K, Nie X, Reed M J, Wang H, Yao G, Zhang M, Chen K. BDS: a centralized near-optimal overlay network for inter-datacenter data replication. In: Proceedings of the 13th EuroSys Conference. 2018, 10

[39] Park J W, Tumanov A, Jiang A, Kozuch M A, Ganger G R. 3Sigma: distribution-based cluster scheduling for runtime uncertainty. In: Proceedings of the 13th EuroSys Conference. 2018, 2

[40] Zheng L, Yang Y, Hauptmann A G. Person re-identification: past, present and future. 2016, arXiv preprint arXiv: 1610.02984

[41] Gulati A, Merchant A, Varman P J. mClock: handling throughput variability for hypervisor IO scheduling. In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation. 2010, 437−450

[42] Li J, Xia Y, Li B, Zeng Z . A pseudo-dynamic search ant colony optimization algorithm with improved negative feedback mechanism. Cognitive Systems Research, 2020, 62: 1–9

[43] Ahmad E S. Infrastructure as a service: a practical study of alibaba cloud elastic compute service (ECS)[J]. Tartous University-A Project, 2019.

[44] GB/T 7714Axboe J. Fio-flexible i/o tester synthetic benchmark. URL, See github. com/axboe/fio website (Accessed: 2015-06-13), 2005

[45] Mittal N, Garg K, Ameria A . A paper on modified round robin algorithm. International Journal of Latest Technology in Engineering, Management & Applied Science, 2015, 4( 11): 93–98

[46] Mdtest hpc benchmark, available from the website of mdtest.sourceforge.net/

Acknowledgements

The work described in this paper are supported by the National Natural Science Foundation of China (Grant Nos. 62104014, 62272026), the National Laboratory of Software Development Environment (No. SKLSDE-2022ZX-07), the Hebei Youth Talents Support Project (No. BJ2019008), and the Natural Science Foundation of Hebei Province (No. F2020204003).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

RIGHTS & PERMISSIONS

2024 Higher Education Press