On Learning Adaptive Service Compositions

Abstract

Service composition is an important and effective technique that enables atomic services to be combined together to forma more powerful service, i.e., a composite service. With the pervasiveness of the Internet and the proliferation of interconnected computing devices, it is essential that service composition embraces an adaptive service provisioning perspective. Reinforcement learning has emerged as a powerful tool to compose and adapt Web services in open and dynamic environments. However, the most common applications of reinforcement learning algorithms are relatively inefficient in their use of the interaction experience data, whichmay affect the stability of the learning process when deployed to cloud environments. In particular, they make just one learning update for each interaction experience. This paper introduces a novel approach that aims to achieve greater data efficiency by saving the experience data and using it in aggregate to make updates to the learned policy. The proposed approach devises an offline learning scheme for cloud service composition where the online learning task is transformed into a series of supervised learning tasks. A set of algorithms is proposed under this scheme in order to facilitate and empower efficient service composition in the cloud under various policies and different scenarios. The results of our experiments show the effectiveness of the proposed approach for composing and adapting cloud services, especially under dynamic environment settings, compared to their online learning counterparts.

Keywords

Service composition / reinforcement learning / cloud services / offline learning

References

[1] Achbany Y, Jureta I J, Faulkner S, Fouss F. Continually learning optimal allocations of services to tasks. IEEE Transactions on Services Computing, 2008, 1(3): 141-154.

[2] Al-Masri E, Mahmoud Q H. Discovering the best web service. Proceedings of WWW, 2007

[3] Al Ridhawi Y, Karmouch A. Decentralized plan-free semantic-based service composition in mobile networks. IEEE Transactions on Services Computing, 2015, 8(1): 17-31.

[4] Baladron C, Aguiar J M, Carro B, Calavia L, Cadenas A, Sanchez-Esguevillas A. Framework for intelligent service adaptation to user’s context in next generation networks. IEEE Communications Magazine, 2012, 50(3): 18-25.

[5] Chiu D, Agrawal G. Cost and accuracy aware scientific workflow composition for service-oriented environments. IEEE Transactions on Services Computing, 2012, 4(2): 140-152.

[6] Ding Z H, Jiang M Y, Kandel A. Port-based reliability computing for service composition. IEEE Transactions on Services Computing, 2012, 5(3): 422-436.

[7] Fernandez H, Tedeschi C, Priol T. A chemistry-inspired workflow management system for a decentralised workflow execution. IEEE Transactions on Services Computing, 2013

[8] Garcia Llinas G A, Nagi R. Network and QoS-based selection of complementary services. IEEE Transactions on Services Computing, 2015, 8(1): 79-91.

[9] Hang C W, Kalia A K, Singh M P. Behind the curtain: Service selection via trust in composite services. Proceedings IEEE ICWS, 2012

[10] Hatzi O, Vrakas D, Nikolaidou M, Bassiliades N, Anagnostopoulos D, Vlahavas I. An integrated approach to automated semanticweb service composition through planning. IEEE Transactions on Services Computing, 2011

[11] Jiang W, Hu S, Lee D, Gong S, Liu Z. Continuous query for QoS-aware automatic service composition. Proceedings of IEEE ICWS, 2012

[12] Lee C H, Hwang S Y, Yen I L. A service pattern model for flexible service composition. Proceedings of IEEE ICWS, 2012

[13] Lee J W. Stock price prediction using reinforcement learning. Proceedings of IEEE ISIE, 2001

[14] Li H, Dagli C H, Enke D. Short-term stock market timing prediction under reinforcement learning schemes. Proceedings of IEEE ADPRL, 2007

[15] Moustafa A, Zhang M. Towards proactive web service adaptation. Proceedings of CAISE, 2012

[16] Moustafa A, Zhang M. Multi-objective service composition using reinforcement learning. Proceedings of ICSOC, Lecture Notes in Computer Science, 2013

[17] Sim K M. Agent based cloud computing. IEEE Transactions on Services Computing, 2012, 5(4): 564-577.

[18] Sutton R S, Barto A G. Introduction to Reinforcement Learning, 1998, 1ed Cambridge, MA, USA: MIT Press.

[19] Tang H, Liu W, Zhou L. Web service composition method using hierarchical reinforcement learning. Proceedings of GCN, 2012

[20] Wada H, Suzuki J, Yamano Y, Oba K. E3: A multi-objective optimisation framework for SLA-aware service composition. IEEE Transactions on Services Computing, 2012, 5(3): 358-372.

[21] Wang H, Chen X, Wu Q, Yu Q, Zheng Z, Bouguettaya A. Integrating on-policy reinforcement learning with multi-agent techniques for adaptive service composition. Proceedings of ICSOC, 2014, Berlin Heidelberg: Springer

[22] Wang H, Wang X. A novel approach to large-scale services composition. Proceedings of APWeb, 2013

[23] Wang H, Zhou X, Zhou X, Liu W, Li W, Bouguettaya A. Adaptive service composition based on reinforcement learning. Proceedings of ICSOC, 2010

[24] Watkins C. Learning from delayed rewards, 1989, England: Cambridge University

[25] Yu Q, Bouguettaya A. Efficient service skyline computation for composite service selection. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(4): 776-789.

[26] Zhang H, Chai H, Zhao W, Melliar-Smith P M, Moser L E. Trustworthy coordination ofweb services atomic transactions. IEEE Transactions on Parallel and Distributed Systems, 2012, 23(8): 1551-1565.

[27] Zheng Z, Hao M, Lyu M R, King I. QoS-aware web service recommendation by collaborative filtering. IEEE Transactions on Services Computing, 2011, 4(2): 140-152.