Towards application-level elasticity on shared cluster: an actor-based approach

Abstract

In current cluster computing, several distributed frameworks are designed to support elasticity for business services adapting to environment fluctuation. However, most existing works support elasticity mainly at the resource level, leaving application level elasticity support problem to domain-specific frameworks and applications. This paper proposes an actor-based general approach to support application-level elasticity for multiple cluster computing frameworks. The actor model offers scalability and decouples language-level concurrency from the runtime environment. By extending actors, a new middle layer called Unisupervisor is designed to “sit” between the resource management layer and application framework layer. Actors in Unisupervisor can automatically distribute and execute tasks over clusters and dynamically scale in/out. Based on Unisupervisor, high-level profiles (MasterSlave, MapReduce, Streaming, Graph, and Pipeline) for diverse cluster computing requirements can be supported. The entire approach is implemented in a prototype system called UniAS. In the evaluation, both benchmarks and real applications are tested and analyzed in a small scale cluster. Results show that UniAS is expressive and efficiently elastic.

Keywords

elasticity / elastic scaling / actor programming model / cluster computing / concurrent and parallel processing

References

[1] LuX C, WangH M, WangJ, Xu J, LiD S .Internet-based virtual computing environment: beyond the data center as a computer. Future Generation Computer Systems, 2013, 29(1): 309–322

[2] ToshniwalA, TanejaS, ShuklaA, Ramasamy K, PatelJ M , KulkarniS, Jackson J, GadeK , FuM S, DonhamJ, BhagatN, Mittal S, RyaboyD . Storm@twitter. In:Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. 2014, 147–156

[3] HindmanB, Konwinski A, ZahariaM , GhodsiA, JosephA D, KatzR, Shenker S, StoicaI . Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. 2011, 295–308

[4] VavilapalliV K, MurthyA C, DouglasC, Agarwal S, KonarM , EvansR, GravesT, LoweJ, Shah H, SethS , SahaB, CurinoC, O’MalleyO , RadiaS, ReedB, BaldeschwielerE . Apache Hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013

[5] ColeM. Algorithmic Skeletons: Structured Management of Parallel Computation. Cambridge, Massachusetts: MIT Press, 1991

[6] BahgaA, Madisetti V K. Rapid prototyping of multitier cloud-based services and systems. Computer, 2013, 46(11): 76–83

[7] KächeleS, HauckF J. Component-based scalability for cloud applications. In: Proceedings of the 3rd International Workshop on Cloud Data and Platforms. 2013, 19–24

[8] CaromelD, LeytonM. Fine tuning algorithmic skeletons. Lecture Notes in Computer Science, vol 4641. Berlin: Springer-Verlag, 2007, 72–81

[9] AghaG A. Actors: a model of concurrent computation in distributed systems. Dissertation for the doctoral Degree. Cambridge: Massachusetts Institute of Technology, 1985

[10] ZhanH L, KangL H, CaoD G. DETS: a dynamic and elastic task scheduler supporting multiple parallel schemes. In: Proceedings of the 8th IEEE International Symposium on Service Oriented System Engineering. 2014, 278–283

[11] MateiZharia, DasT, LiH Y, Hunter T, ShenkerS , StoicaI. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles (OSDI). 2013, 423–438

[12] LinJ, ZhaL, XuZ W. Consolidated cluster systems for data centers in the cloud age: a survey and analysis. Frontiers of Computer Science, 2013, 7(1): 1–19

[13] LiuL. Computing infrastructure for big data processing. Frontiers of Computer Science, 2013, 7(2): 165–170

[14] ZachariaF, Govindaraju M.Delma: dynamically elastic mapreduce framework for CPU-intensive applications. In: Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2011, 454–463

[15] GordonA W, LuP. Elastic phoenix: malleable mapreduce for sharedmemory systems. Network and Parallel Computing, Vol 6985. Berlin: Springer, 2011

[16] ShiW, HongB. Clotho: an elastic MapReduce workload/runtime codesign. In: Proceedings of the 12th International Workshop on Adaptive and Reflective Middleware. 2013

[17] WaldemarH, Satzger B, DustdarS . Elastic stream processing in the Cloud. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2013, 3(5): 333–345

[18] QianZ, HeY, SuC Z, Wu Z J, ZhuH Y , ZhangT Z, ZhouL D, YuY, ZhangZ. Timestream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14

[19] BugraG, Schneider S, HirzelM , WuK L. Elastic scaling for data stream processing. IEEE Transcations on Parallel and Distributed Systems, 2014, 25(6): 1447–1463

[20] HuangC, ZhengG B, KaleL, Kumar S. Performance evaluation of adaptive MPI. In: Proceedings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2006, 12–21

[21] WeiY, Sukumar K, VecchiolaC , KarunamoorthyD, BuyyaR. Aneka cloud application platform and its integration with Windows Azure. 2011, arXiv preprint arXiv: 1103.2590

[22] BykovS, GellerA, KliotG, Larus J R, PandyaR , ThelinJ. Orleans: cloud computing for everyone. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg