Please wait a minute...

Frontiers of Computer Science

Front. Comput. Sci.    2017, Vol. 11 Issue (5) : 803-820     DOI: 10.1007/s11704-016-5386-9
Towards application-level elasticity on shared cluster: an actor-based approach
Donggang CAO1,2(), Lianghuan KANG1,2, Hanglong ZHAN1,2, Hong MEI1
1. Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing 100871, China
2. Beida (Binhai) Information Research, Tianjin 300450, China
Download: PDF(1308 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

In current cluster computing, several distributed frameworks are designed to support elasticity for business services adapting to environment fluctuation. However, most existing works support elasticity mainly at the resource level, leaving application level elasticity support problem to domain-specific frameworks and applications. This paper proposes an actor-based general approach to support application-level elasticity for multiple cluster computing frameworks. The actor model offers scalability and decouples language-level concurrency from the runtime environment. By extending actors, a new middle layer called Unisupervisor is designed to “sit” between the resource management layer and application framework layer. Actors in Unisupervisor can automatically distribute and execute tasks over clusters and dynamically scale in/out. Based on Unisupervisor, high-level profiles (MasterSlave, MapReduce, Streaming, Graph, and Pipeline) for diverse cluster computing requirements can be supported. The entire approach is implemented in a prototype system called UniAS. In the evaluation, both benchmarks and real applications are tested and analyzed in a small scale cluster. Results show that UniAS is expressive and efficiently elastic.

Keywords elasticity      elastic scaling      actor programming model      cluster computing      concurrent and parallel processing     
Corresponding Authors: Donggang CAO   
Just Accepted Date: 25 February 2016   Online First Date: 31 October 2016    Issue Date: 26 September 2017
 Cite this article:   
Donggang CAO,Lianghuan KANG,Hanglong ZHAN, et al. Towards application-level elasticity on shared cluster: an actor-based approach[J]. Front. Comput. Sci., 2017, 11(5): 803-820.
E-mail this article
E-mail Alert
Articles by authors
Donggang CAO
Lianghuan KANG
Hanglong ZHAN
Hong MEI
1 LuX C, WangH M, WangJ, Xu J, LiD S .Internet-based virtual computing environment: beyond the data center as a computer. Future Generation Computer Systems, 2013, 29(1): 309–322
doi: 10.1016/j.future.2011.08.005
2 ToshniwalA, TanejaS, ShuklaA, Ramasamy K, PatelJ M , KulkarniS, Jackson J, GadeK , FuM S, DonhamJ, BhagatN, Mittal S, RyaboyD . Storm@twitter. In:Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. 2014, 147–156
doi: 10.1145/2588555.2595641
3 HindmanB, Konwinski A, ZahariaM , GhodsiA, JosephA D, KatzR, Shenker S, StoicaI . Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. 2011, 295–308
4 VavilapalliV K, MurthyA C, DouglasC, Agarwal S, KonarM , EvansR, GravesT, LoweJ, Shah H, SethS , SahaB, CurinoC, O’MalleyO , RadiaS, ReedB, BaldeschwielerE . Apache Hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013
doi: 10.1145/2523616.2523633
5 ColeM. Algorithmic Skeletons: Structured Management of Parallel Computation. Cambridge, Massachusetts: MIT Press, 1991
6 BahgaA, Madisetti V K. Rapid prototyping of multitier cloud-based services and systems. Computer, 2013, 46(11): 76–83
doi: 10.1109/MC.2013.154
7 KächeleS, HauckF J. Component-based scalability for cloud applications. In: Proceedings of the 3rd International Workshop on Cloud Data and Platforms. 2013, 19–24
doi: 10.1145/2460756.2460760
8 CaromelD, LeytonM. Fine tuning algorithmic skeletons. Lecture Notes in Computer Science, vol 4641. Berlin: Springer-Verlag, 2007, 72–81
doi: 10.1007/978-3-540-74466-5_9
9 AghaG A. Actors: a model of concurrent computation in distributed systems. Dissertation for the doctoral Degree. Cambridge: Massachusetts Institute of Technology, 1985
10 ZhanH L, KangL H, CaoD G. DETS: a dynamic and elastic task scheduler supporting multiple parallel schemes. In: Proceedings of the 8th IEEE International Symposium on Service Oriented System Engineering. 2014, 278–283
doi: 10.1109/sose.2014.39
11 MateiZharia, DasT, LiH Y, Hunter T, ShenkerS , StoicaI. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles (OSDI). 2013, 423–438
doi: 10.1145/2517349.2522737
12 LinJ, ZhaL, XuZ W. Consolidated cluster systems for data centers in the cloud age: a survey and analysis. Frontiers of Computer Science, 2013, 7(1): 1–19
doi: 10.1007/s11704-012-2086-y
13 LiuL. Computing infrastructure for big data processing. Frontiers of Computer Science, 2013, 7(2): 165–170
doi: 10.1007/s11704-013-3900-x
14 ZachariaF, Govindaraju M.Delma: dynamically elastic mapreduce framework for CPU-intensive applications. In: Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2011, 454–463
15 GordonA W, LuP. Elastic phoenix: malleable mapreduce for sharedmemory systems. Network and Parallel Computing, Vol 6985. Berlin: Springer, 2011
16 ShiW, HongB. Clotho: an elastic MapReduce workload/runtime codesign. In: Proceedings of the 12th International Workshop on Adaptive and Reflective Middleware. 2013
doi: 10.1145/2541583.2541588
17 WaldemarH, Satzger B, DustdarS . Elastic stream processing in the Cloud. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2013, 3(5): 333–345
doi: 10.1002/widm.1100
18 QianZ, HeY, SuC Z, Wu Z J, ZhuH Y , ZhangT Z, ZhouL D, YuY, ZhangZ. Timestream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14
doi: 10.1145/2465351.2465353
19 BugraG, Schneider S, HirzelM , WuK L. Elastic scaling for data stream processing. IEEE Transcations on Parallel and Distributed Systems, 2014, 25(6): 1447–1463
doi: 10.1109/TPDS.2013.295
20 HuangC, ZhengG B, KaleL, Kumar S. Performance evaluation of adaptive MPI. In: Proceedings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2006, 12–21
doi: 10.1145/1122971.1122976
21 WeiY, Sukumar K, VecchiolaC , KarunamoorthyD, BuyyaR. Aneka cloud application platform and its integration with Windows Azure. 2011, arXiv preprint arXiv: 1103.2590
[1] FCS-0803-15386-DGC_suppl_1 Download
Full text