An adaptive switching scheme for iterative computing in the cloud

Abstract

Delta-based accumulative iterative computation (DAIC) model is currently proposed to support iterative algorithms in a synchronous or an asynchronous way. However, both the synchronous DAIC model and the asynchronous DAIC model only satisfy some given conditions, respectively, and perform poorly under other conditions either for high synchronization cost or for many redundant activations. As a result, the whole performance of both DAIC models suffers fromthe serious network jitter and load jitter caused bymultitenancy in the cloud. In this paper, we develop a system, namely HybIter, to guarantee the performance of iterative algorithms under different conditions. Through an adaptive execution model selection scheme, it can efficiently switch between synchronous and asynchronous DAIC model in order to be adapted to different conditions, always getting the best performance in the cloud. Experimental results show that our approach can improve the performance of current solutions up to 39.0%.

Keywords

iterative algorithm / computational skew / communication skew / cloud / delta-based accumulative iterative computation

References

[1] Horowitz D, Kamvar S D. The anatomy of a large-scale social search engine. In: Proceedings of the 19th International Conference on World Wide Web. 2010, 431−440

[2] Song H, Cho T, Dave V, Zhang Y, Qiu L. Scalable proximity estimation and link prediction in online social networks. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference. 2009, 322−335

[3] Gao B, Liu T, Wei W, Wang T, Li H. Semi-supervised ranking on very large graphs with rich metadata. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011, 96−104

[4] Baluja S, Seth R, Sivakumar D, Jing Y, Jay Y, Kumar S, Deepak R, Aly M. Video suggestion and discovery for youtube: taking random walks through the view graph. In: Proceedings of the 17th International Conference on World Wide Web. 2008, 895−904

[5] Zhou T, Kuscsik Z, Liu J, Medo M, Wakeling J R, Zhang Y. Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences, 2010, 107(10): 4511−4515

[6] David L N, Jon K. The link prediction problem for social networks. In: Proceedings of the 12th International Conference on Information and Knowledge Management. 2003, 556−559

[7] Shroff G M. A parallel algorithm for the eigenvalues and eigenvectors of a general complex matrix. Numerische Mathematik, 1990, 58(1): 779−805

[8] Ekanayake J, Li H, Zhang B, Gunarathne T, Bae S H, Qiu J, Fox G. Twister: a runtime for iterative MapReduce. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing. 2010, 810−818

[9] Bu Y, Howe B, Balazinska M, Ernst M D. HaLoop: efficient iterative data processing on large clusters. Proceedings of the VLDB Endowment, 2010, 3(1): 285−296

[10] Zaharia M, Chowdhury M, Franklin M J, Shenker S, Stoica I. Spark: cluster computing with working sets. In: Proceedings of the 2^nd USENIX Conference on Hot Topics in Cloud Computing. 2010, 1−10

[11] Low Y, Gonzalez J, Kyrola A, Bickson D, Guestrin C, Hellerstein J M. Graphlab: a new framework for parallel machine learning. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence. 2010, 1−10

[12] Zhang Y, Gao Q, Gao L, Wang C. Maiter: an asynchronous graph processing framework for delta-based accumulative iterative computation. IEEE Transactions on Parallel and Distributed System, 2013, http://doi.ieeecomputersociety.org/10.1109/TPDS.2013.235

[13] Zou T, Wang G, Salles M V, Bindel D, Demers A, Gehrke J, White W. Making time-stepped applications tick in the cloud. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011, 1−20

[14] Zhang Y, Gao Q, Gao L, Wang C. Imapreduce: a distributed computing framework for iterative computation. In: Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum. 2011, 1112−1121

[15] Power R, Li J. Piccolo: building fast, distributed programs with partitioned tables. In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation. 2010, 1−14

[16] Logothetis D, Olston C, Reed B, Webb K C, Yocum K. Stateful bulk processing for incremental analytics. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 51−62

[17] Murray D G, Schwarzkopf M, Smowton C, Smith S, Madhavapeddy A, Hand S. CIEL: a universal execution engine for distributed dataflow computing. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. 2011, 1−9

[18] Malewicz G, Austern M H, Bik A, Dehnert J C, Horn I, Leiser N, Czajkowski G. Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data. 2010, 135−146

[19] Chazan D, Miranker W. Chaotic relaxation. Linear Algebra and Its Applications, 1969, 2(2): 199−222

[20] Baudet G M. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 1978, 25(2): 226−244

[21] Bertsekas D P. Distributed asynchronous computation of fixed points. Mathematical Programming, 1983, 27(1): 107−120

[22] Kambatla K, Rapolu N, Jagannathan S, Grama A. Asynchronous algorithms in mapreduce. In: Proceedings of the 2010 IEEE International Conference on Cluster Computing. 2010, 245−254

[23] Low Y, Bickson D, Gonzalez J, Guestrin C, Kyrola A, Hellerstein J M. Distributed GraphLab: a framework for machine learning and data mining in the cloud. Proceedings of the VLDB Endowment, 2012, 5(8): 716−727

[24] Zhang Y, Gao Q, Gao L, Wang C. Accelerate large-scale iterative computation through asynchronous accumulative updates. In: Proceedings of the 3rdWorkshop on Scientific Cloud Computing Date. 2012, 13−22

[25] Stanford. Stanford Large Network Dataset Collection. http://snap.stanford.edu/data/, 2013

[26] Takács G, Pilászy I, Németh B, Tikk D. Scalable collaborative filtering approaches for large recommender systems. The Journal of Machine Learning Research, 2009, 10: 623−656

[27] Haewoon K. What is Twitter, a Social Network or a New Media? 2013

[28] Zhang Y, Gao Q, Gao L, Wang C. PrIter: a distributed framework for prioritized iterative computations. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011, 1−14

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg