HWM: a hybrid workload migration mechanism of metadata server cluster in data center

Abstract

In data center, applications of big data analytics pose a big challenge to massive storage systems. It is significant to achieve high availability, high performance and high scalability for PB-scale or EB-scale storage systems. Metadata server (MDS) cluster architecture is one of the most effective solutions to meet the requirements of applications in data center. Workload migration can achieve load balance and energy saving of cluster systems. In this paper, a hybrid workload migration mechanism of MDS cluster is proposed and named as HWM. In HWM, workload of MDS is classified into two categories: metadata service and state service, and they can be migrated rapidly from a source MDS to a target MDS in different ways. Firstly, in metadata service migration, all the dirty metadata of one sub file system is flushed to a shared storage pool by the source MDS, and then is loaded by the target MDS. Secondly, in state service migration, all the states of that sub file system are migrated from source MDS to target MDS through network at file granularity, and then all of the related structures of these states are reconstructed in targetMDS. Thirdly, in the process of workload migration, instead of blocking client requests, the source MDS can decide which MDS will respond to each request according to the operation type and the migration stage. The proposed mechanismis implemented in the BlueWhaleMDS cluster. The performance measurements show that the HWM mechanism is efficient to migrate the workload of a MDS cluster system and provides low-latency access to metadata and states.

Keywords

data center / metadata server cluster / hybrid workload migration / metadata service / state service / lowlatency access

References

[1] Turner V, Gantz J F, Reinsel D, Minton S. The digital universe of opportunities: rich data and the increasing value of the Internet of things. IDC Analyze the Future, 2014

[2] Ghemawat S, Gobioff H, Leung S T. The Google file system. ACM SIGOPS Operating Systems Review, 2003, 37(5): 29–43

[3] McKusick K, Quinlan S. GFS: evolution on fast-forward. Communications of the ACM, 2010, 53(3): 42–49

[4] Makoto S, Hiroki K, Shoji K. Performance evaluation of scaleout NAS for HDFS. In: Proceedings of the 3rd International Conference on Advances in Information Mining and Management. 2013, 32–35

[5] Xia M, Saxena M, Blaum M, Pease D A. A tale of two erasure codes in HDFS. In: Proceedings of the 13th USENIX Conference on File and Storage Technologies. 2015, 213–226

[6] Jain R, Sarkar P, Subhraveti D. GPFS-SNC: an enterprise cluster file system for big data. IBM Journal of Research and Development, 2013, 57(3/4): 5:1–5:10

[7] Davies A, Orsaria A. Scale out with GlusterFS. Linux Journal, 2013, (235): 1

[8] Chasapis K, Dolz M F, Kuhn M, Ludwig T. Evaluating Lustre’s metadata server on a multi-socket platform. In: Proceedings of the 9th Parallel Data Storage Workshop. 2014, 13–18

[9] Kim T, Noh S H. PNFS for everyone: an empirical study of a low-cost, highly scalable networked storage. International Journal of Computer Science and Network Security, 2014, 14(3): 52–59

[10] Weil S A, Brandt S A, Miller E L, Long D D, Maltzahn C. Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation. 2006, 307–320

[11] Wang F, Nelson M, Oral S, Atchley S, Weil S, Settlemyer B W, Caldwell B, Hill J. Performance and scalability evaluation of the Ceph parallel file system. In: Proceedings of the 8th Parallel Data StorageWorkshop. 2013, 14–19

[12] Sevilla M A, Watkins N, Maltzahn C, Nassi I, Brandt S A, Weil S A, Farnum G, Fineberg S. Mantle: a programmable metadata load balancer for the Ceph file system. In: Proceedings of the 27th International Conference for High Performance Computing, Networking, Storage and Analysis. 2015, 1–12

[13] Sinnamohideen S, Sambasivan R R, Hendricks J, Liu L, Ganger G R. A transparently-scalable metadata service for the UrsaMinor storage system. In: Proceedings of USENIX Annual Technical Conference. 2010, 13–26

[14] Abd-El-Malek M, Courtright IIWV, Cranor C, Ganger G R, Hendricks J, Klosterman A J, Mesnier M P, Prasad M, Salmon B, Sambasivan R R, S Sinnamohideen, Strunk J D, Thereska E, Wachs M, Wylie J J. Ursa Minor: versatile cluster-based storage. In: Proceedings of the 4th USENIX Conference on File and Storage Technologies. 2005, 59–72

[15] Menon J, Pease D A, Rees R, Duyanovich L, Hillsberg B. IBM Storage Tank—a heterogeneous scalable SAN file system. IBM Systems Journal, 2003, 42(2): 250–267

[16] Thomasian A. Storage research in industry and universities. ACM SIGARCH Computer Architecture News, 2010, 38(2): 1–48

[17] An overview of NFSv4: NFSv4.0, NFSv4.1, pNFS, and proposed NFSv4.2 features. SNIA Ethernet Storage Forum, 2012

[18] Mohr R, Peltz Jr P. Benchmarking SSD-based Lustre file system configurations. In: Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment. 2014

[19] Aishwarya K, Sreevatson M, Babu C, Prabavathy B. Efficient prefetching technique for storage of heterogeneous small files in hadoop dis tributed file system federation. In: Proceedings of the 15th International Conference on Advanced Computing. 2013, 523–530

[20] Chen G, Jagadish H, Jiang D, Maier D, Ooi B C, Tan K L, Tan W C. Federation in cloud data management: challenges and opportunities. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1670–1678

[21] Patil S, Gibson G A. Scale and concurrency of GIGA+: file system directories with millions of files. In: Proceedings of the 9th USENIX Conference on File and Storage Technologies. 2011, 1–14

[22] Patil S V, Gibson G A, Lang S, Polte M. GIGA+: scalable directories for shared file systems. In: Proceedings of the 2nd International Workshop on Petascale Data Storage. 2007, 26–29

[23] Douceur J R, Howell J. Distributed directory service in the Farsite file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation. 2006, 321–334

[24] Ma H, Liu Z, Zhang H, Feng S, Han X, Xu L. Experiences with hierarchical storage management support in blue whale file system. In: Proceedings of the 11th International Conference on Parallel and Distributed Computing, Applications and Technologies. 2010, 369–374

[25] Solar R, Gil-Costa V, Marin M. Dynamic load balance for approximate parallel simulations with consistent hashing. In: Proceedings of the 47th Summer Computer Simulation Conference. 2015, 1–10

[26] Xu Z Y, Wang X X. A predictive modified round robin scheduling algorithm for Web server clusters. In: Proceedings of the 34th Chinese Control Conference. 2015, 5804–5808

[27] Xia Y, Dobra A, Han S C. Multiple-choice random network for server load balancing. In: Proceedings of the 26th IEEE International Conference on Computer Communications. 2007, 1982–1990

[28] Wu Y, Luo S, Li Q. An adaptive weighted least-load balancing algorithm based on server cluster. In: Proceedings of the 5th International Conference on Intelligent Human-Machine Systems and Cybernetics. 2013, 224–227

[29] Allayear S M, Salahuddin M, Ahmed F, Park S S. Introducing iSCSI protocol on online based mapreduce mechanism. In: Proceedings of the 14th International Conference on Computational Science and Its Applications. 2014, 691–706

[30] Guo T, Shen Y L, Liu Z J, Xu L. BW-FILERAID: a kind of file based distributed RAID system and optimization. Applied Mechanics and Materials, 2011, 80: 1208–1216

[31] Chen Y, Wu F, Yu K, Zhang L, Chen Y, Yang Y, Mao J. Instant bug testing service for linux kernel. In: Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications. 2013, 1860–1865

RIGHTS & PERMISSIONS

2016 Higher Education Press and Springer-Verlag Berlin Heidelberg