Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities

Abstract

With the demand of agile development and management, cloud applications today are moving towards a more fine-grained microservice paradigm, where smaller and simpler functioning parts are combined for providing end-to-end services. In recent years, we have witnessed many research efforts that strive to optimize the performance of cloud computing system in this new era. This paper provides an overview of existing works on recent system performance optimization techniques and classify them based on their design focuses. We also identify open issues and challenges in this important research direction.

Graphical abstract

Keywords

microservice / cloud computing system / performance optimization / challenges / opportunities

References

[1] Gan Y, Zhang Y, Hu K, Cheng D, He Y, Pancholi M, Delimitrou C. Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 19–33

[2] Chen Y, Luo T, Liu S, Zhang S, He L, Wang J, Li L, Chen T, Xu Z, Sun N, Temam O. DaDianNao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture. 2014, 609– 622

[3] Jouppi N P, Young C, Patil N, Patterson D, Agrawal G. In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture. 2017, 1– 12

[4] Chung E , Fowers J , Ovtcharov K , Papamichael M , Caulfield A . Serving DNNs in real time at datacenter scale with project brainwave. IEEE Micro, 2018, 38( 2): 8– 20

[5] Nitu V, Teabe B, Tchana A, Isci C, Hagimont D. Welcome to zombieland: practical and energy-efficient memory disaggregation in a datacenter. In: Proceedings of the 13th EuroSys Conference. 2018, 16

[6] Lim K, Chang J, Mudge T, Ranganathan P, Reinhardt S K, Wenisch T. Disaggregated memory for expansion and sharing in blade servers. In: Proceedings of the 36th Annual International Symposium on Computer Architecture. 2009, 267– 278

[7] Taibi D, Lenarduzzi V, Pahl C. Architectural patterns for microservices: a systematic mapping study. In: Proceedings of the 8th International Conference on Cloud Computing and Services Science. 2018, 221– 232

[8] Alshuqayran N, Ali N, Evans R. A systematic mapping study in microservice architecture. In: Proceedings of the 9th IEEE International Conference on Service-Oriented Computing and Applications. 2016, 44–51

[9] Aguiar L, Almeida W, Hazin R, Lima A, Ferraz F. Survey on microservice architecture-security, privacy and standardization on cloud computing environment. In: Proceedings of the 12th International Conference on Software Engineering Advances. 2017, 210

[10] Yarygina T, Bagge A B. Overcoming security challenges in microservice architectures. In: Proceedings of 2018 IEEE Symposium on Service-Oriented System Engineering. 2018, 11– 20

[11] Villamizar M, Garcés O, Castro H, Verano M, Salamanca L, Casallas R, Gil S. Evaluating the monolithic and the microservice architecture pattern to deploy Web applications in the cloud. In: Proceedings of the 10th Computing Colombian Conference. 2015, 583–590

[12] Vural H, Koyuncu M, Guney S. A systematic literature review on microservices. In: Proceedings of the 17th International Conference on Computational Science and its Applications. 2017, 203– 217

[13] Gouigoux J P, Tamzalit D. From monolith to microservices: lessons learned on an industrial migration to a Web oriented architecture. In: Proceedings of 2017 IEEE International Conference on Software Architecture Workshops. 2017, 62–65

[14] Di Francesco P, Lago P, Malavolta I. Migrating towards microservice architectures: an industrial survey. In: Proceedings of 2018 IEEE International Conference on Software Architecture. 2018, 29– 2909

[15] Manvi S S , Shyam G K . Resource management for Infrastructure as a Service (IaaS) in cloud computing: a survey. Journal of Network and Computer Applications, 2014, 41 : 424– 440

[16] Vaquero L M , Cuadrado F , Elkhatib Y , Bernal-Bernabe J , Srirama S N , Zhani M F . Research challenges in nextgen service orchestration. Future Generation Computer Systems, 2019, 90 : 20– 38

[17] Pahl C, Jamshidi P. Microservices: a systematic mapping study. In: Proceedings of the 6th International Conference on Cloud Computing and Services Science. 2016, 137– 146

[18] Hassan S, Bahsoon R. Microservices and their design trade-offs: a self-adaptive roadmap. In: Proceedings of 2016 IEEE International Conference on Services Computing. 2016, 813– 818

[19] Toffetti G, Brunner S, Blöchlinger M, Dudouet F, Edmonds A. An architecture for self-managing microservices. In: Proceedings of the 1st International Workshop on Automated Incident Management in Cloud. 2015, 19– 24

[20] Familiar B. Microservices, IoT, and Azure: Leveraging DevOps and Microservice Architecture to Deliver SaaS Solutions. Berkeley: Apress, 2015

[21] Jamshidi P , Pahl C , Mendonça N C , Lewis J , Tilkov S . Microservices: the journey so far and challenges ahead. IEEE Software, 2018, 35( 3): 24– 35

[22] Baldini I, Castro P, Chang K, Cheng P, Fink S, Ishakian V, Mitchell N, Muthusamy V, Rabbah R, Slominski A, Suter P. Serverless computing: current trends and open problems. In: Chaudhary S, Somani G, Buyya R, eds. Research Advances in Cloud Computing. Singapore: Springer, 2017, 1– 20

[23] Fox G C, Ishakian V, Muthusamy V, Slominski A. Status of serverless computing and function-as-a-service (FaaS) in industry and research. 2017, arXiv preprint arXiv: 1708.08028

[24] Castro P, Ishakian V, Muthusamy V, Slominski A. Serverless programming (function as a service). In: Proceedings of the IEEE 37th International Conference on Distributed Computing Systems. 2017, 2658−2659

[25] Yan M, Castro P, Cheng P, Ishakian V. Building a chatbot with serverless computing. In: Proceedings of the 1st International Workshop on Mashups of Things and APIs. 2016, 5

[26] Ishakian V, Muthusamy V, Slominski A. Serving deep learning models in a serverless platform. In: Proceedings of 2018 IEEE International Conference on Cloud Engineering. 2018, 257– 262

[27] Castro P , Ishakian V , Muthusamy V , Slominski A . The rise of serverless computing. Communications of the ACM, 2019, 62( 12): 44– 54

[28] Kritikos K, Skrzypek P. A review of serverless frameworks. In: Proceedings of IEEE/ACM International Conference on Utility and Cloud Computing Companion. 2018, 161– 168

[29] Michelson B M . Event-driven architecture overview. Patricia Seybold Group, 2006, 2( 12): 10– 1571

[30] Thalheim J, Rodrigues A, Akkus I E, Bhatotia P, Chen R, Viswanath B, Jiao L, Fetzer C. Sieve: actionable insights from monitored metrics in distributed systems. In: Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference. 2017, 14– 27

[31] Cui W, Richins D, Zhu Y, Reddi V J. Tail latency in node.js: energy efficient turbo boosting for long latency requests in event-driven web services. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2019, 152– 164

[32] Terzic B, Dimitrieski V, Kordić S, Luković I. A model-driven approach to microservice software architecture establishment. In: Proceedings of 2018 Federated Conference on Computer Science and Information Systems. 2018, 73– 80

[33] Rademacher F, Sachweh S, Zündorf A. Differences between model-driven development of service-oriented and microservice architecture. In: Proceedings of 2017 IEEE International Conference on Software Architecture Workshops. 2017, 38– 45

[34] Mellor S, Scott K, Uhl A, Weise D. Model-driven architecture. In: Proceedings of International Conference on Object-Oriented Information Systems. 2002, 290– 297

[35] Da Silva A R . Model-driven engineering: a survey supported by the unified conceptual model. Computer Languages, Systems & Structures, 2015, 43 : 139– 155

[36] Seidewitz E . What models mean. IEEE Software, 2003, 20( 5): 26– 32

[37] Vale S, Hammoudi S. Model driven development of context-aware service oriented architecture. In: Proceedings of the 11th IEEE International Conference on Computational Science and Engineering-Workshops. 2008, 412– 418

[38] Ameller D , Burgués X , Collell O , Costal D , Franch X , Papazoglou M P . Development of service-oriented architectures using model-driven development: a mapping study. Information and Software Technology, 2015, 62 : 42– 66

[39] Fazio M , Celesti A , Ranjan R , Liu C , Chen L , Villari M . Open issues in scheduling microservices in the cloud. IEEE Cloud Computing, 2016, 3( 5): 81– 88

[40] Zhou X, Peng X, Xie T, Sun J, Xu C, Ji C, Zhao W. Benchmarking microservice systems for software engineering research. In: Proceedings of the 40th IEEE/ACM International Conference on Software Engineering: Companion. 2018, 323–324

[41] Aderaldo C M, Mendonça N C, Pahl C, Jamshidi P. Benchmark requirements for microservices architecture research. In: Proceedings of the 1st IEEE/ACM International Workshop on Establishing the Community-Wide Infrastructure for Architecture-Based Software Engineering. 2017, 8–13

[42] Zhou X , Peng X , Xie T , Sun J , Ji C , Li W , Ding D . Fault analysis and debugging of microservice systems: industrial survey, benchmark system, and empirical study. IEEE Transactions on Software Engineering, 2021, 47( 2): 243– 260

[43] Gan Y, Zhang Y, Cheng D, Shetty A, Rathi P, et al. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 3−18

[44] Sriraman A, Wenisch T F. µ suite: a benchmark suite for microservices. In: Proceedings of 2018 IEEE International Symposium on Workload Characterization. 2018, 1−12

[45] Kratzke N, Quint P C. ppbench-a visualizing network benchmark for microservices. In: Proceedings of the 6th International Conference on Cloud Computing and Services Science. 2016, 223−231

[46] Sriraman A, Wenisch T F. µtune: auto-tuned threading for OLDI microservices. In: Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation. 2018, 177−194

[47] Papapanagiotou I, Chella V. NDBench: benchmarking microservices at scale. 2018, arXiv preprint arXiv: 1807.10792

[48] Ueda T, Nakaike T, Ohara M. Workload characterization for microservices. In: Proceedings of 2016 IEEE International Symposium on Workload Characterization. 2016, 1−10

[49] Gan Y , Delimitrou C . The architectural implications of cloud microservices. IEEE Computer Architecture Letters, 2018, 17( 2): 155– 158

[50] Sriraman A, Dhanotia A, Wenisch T F. SoftSKU: optimizing server architectures for microservice diversity @scale. In: Proceedings of the 46th International Symposium on Computer Architecture. 2019, 513−526

[51] Liu L. Qos-aware machine learning-based multiple resources scheduling for microservices in cloud environment. 2019, arXiv preprint arXiv: 1911.13208

[52] Xavier B, Ferreto T, Jersak L. Time provisioning evaluation of KVM, docker and unikernels in a cloud platform. In: Proceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2016, 277−280

[53] Saha P, Beltre A, Uminski P, Govindaraju M. Evaluation of docker containers for scientific workloads in the cloud. In: Proceedings of the Practice and Experience on Advanced Research Computing. 2018, 11

[54] Jaramillo D, Nguyen D V, Smart R. Leveraging microservices architecture by using docker technology. In: Proceedings of 2016 IEEE SoutheastCon. 2016, 1−5

[55] Kang H, Le M, Tao S. Container and microservice driven design for cloud infrastructure DevOps. In: Proceedings of 2016 IEEE International Conference on Cloud Engineering. 2016, 202−211

[56] Lynn T, Rosati P, Lejeune A, Emeakaroha V. A preliminary review of enterprise serverless cloud computing (function-as-a-service) platforms. In: Proceedings of 2017 IEEE International Conference on Cloud Computing Technology and Science. 2017, 162−169

[57] Esposito C , Castiglione A , Choo K K R . Challenges in delivering software in the cloud as microservices. IEEE Cloud Computing, 2016, 3( 5): 10– 14

[58] Villamizar M, Garcés O, Ochoa L, Castro H, Salamanca L, Verano M, Casallas R, Gil S, Valencia C, Zambrano A, Lang M. Infrastructure cost comparison of running Web applications in the cloud using AWS lambda and monolithic and microservice architectures. In: Proceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2016, 179−182

[59] Friðriksson V. Container overhead in microservice systems. KTH Royal Institute of Technology, Dissertation, 2018

[60] Amaral M, Polo J, Carrera D, Mohomed I, Unuvar M, Steinder M. Performance evaluation of microservices architectures using containers. In: Proceedings of the 14th IEEE International Symposium on Network Computing and Applications. 2015, 27−34

[61] Kratzke N. About microservices, containers and their underestimated impact on network performance. 2017, arXiv preprint arXiv: 1710.04049

[62] Osses F, Márquez G, Astudillo H. Poster: exploration of academic and industrial evidence about architectural tactics and patterns in microservices. In: Proceedings of the 40th IEEE/ACM International Conference on Software Engineering: Companion. 2018, 256−257

[63] Shadija D, Rezai M, Hill R. Microservices: granularity vs. performance. In: Proceedings of the10th International Conference on Utility and Cloud Computing. 2017, 215−220

[64] Lloyd W, Ramesh S, Chinthalapati S, Ly L, Pallickara S. Serverless computing: an investigation of factors influencing microservice performance. In: Proceedings of 2018 IEEE International Conference on Cloud Engineering. 2018, 159−169

[65] Baek H, Srivastava A, Van der Merwe J. CloudSight: a tenant-oriented transparency framework for cross-layer cloud troubleshooting. In: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2017, 268−273

[66] Da Cunha Rodrigues G, Calheiros R N, Guimaraes V T, Dos Santos G L, De Carvalho M B, Granville L, Tarouco L M R, Buyya R. Monitoring of cloud computing environments: concepts, solutions, trends, and future directions. In: Proceedings of the 31st Annual ACM Symposium on Applied Computing. 2016, 378−383

[67] Nicol J, Li C, Chen P, Feng T, Ramachandra H. ODP: an infrastructure for on-demand service profiling. In: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering. 2018, 139−144

[68] Cinque M, Della Corte R, Pecchia A. Microservices monitoring with event logs and black box execution tracing. IEEE Transactions on Services Computing, 2019, DOI: 10.1109/TSC.2019.2940009

[69] Sambasivan R, Shafer I, Mace J, Sigelman B, Fonseca R, Ganger G R. Principled workflow-centric tracing of distributed systems. In: Proceedings of the 7th ACM Symposium on Cloud Computing. 2016, 401−414

[70] Wu L, Bogatinovski J, Nedelkoski S, Tordsson J, Kao O. Performance diagnosis in cloud microservices using deep learning. In: Proceedings of International Conference on Service-Oriented Computing. 2020, 85– 96

[71] Ravichandiran R, Bannazadeh H, Leon-Garcia A. Anomaly detection using resource behaviour analysis for Autoscaling systems. In: Proceedings of the 4th IEEE Conference on Network Softwarization and Workshops. 2018, 192−196

[72] Brandón Á , Solé M , Huélamo A , Solans D , Pérez M S , Muntés-Mulero V . Graph-based root cause analysis for service-oriented and microservice architectures. Journal of Systems and Software, 2020, 159 : 110432–

[73] Lin J, Chen P, Zheng Z. Microscope: pinpoint performance issues with causal graphs in micro-service environments. In: Proceedings of the 16th International Conference on Service-Oriented Computing. 2018, 3−20

[74] Zhang X, Tune E, Hagmann R, Jnagal R, Gokhale V, Wilkes J. CPI²: CPU performance isolation for shared compute clusters . In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 379−391

[75] Margaritov A, Gupta S, Gonzalez-Alberquilla R, Grot B. Stretch: balancing QoS and throughput for colocated server workloads on SMT cores. In: Proceedings of 2019 IEEE International Symposium on High Performance Computer Architecture. 2019, 15−27

[76] Bao L , Wu C , Bu X , Ren N , Shen M . Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Transactions on Parallel and Distributed Systems, 2019, 30( 9): 2114– 2129

[77] Jindal A, Podolskiy V, Gerndt M. Performance modeling for cloud microservice applications. In: Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering. 2019, 25−32

[78] Khazaei H, Mahmoudi N, Barna C, Litoiu M. Performance modeling of microservice platforms. 2019, arXiv preprint arXiv: 1902.0338

[79] Gribaudo M, Iacono M, Manini D. Performance evaluation of massively distributed microservices based applications. In: Proceedings of the 31st European Conference on Modelling and Simulation. 2017, 598−604

[80] Kannan R S, Subramanian L, Raju A, Ahn J, Mars J, Tang L. GrandSLAm: guaranteeing SLAs for jobs in microservices execution frameworks. In: Proceedings of the 14th EuroSys Conference. 2019, 34

[81] Correia J, Ribeiro F, Filipe R, Arauio F, Cardoso J. Response time characterization of microservice-based systems. In: Proceedings of the 17th IEEE International Symposium on Network Computing and Applications. 2018, 1−5

[82] Yu Y , Yang J , Guo C , Zheng H , He J . Joint optimization of service request routing and instance placement in the microservice system. Journal of Network and Computer Applications, 2019, 147 : 102441–

[83] Yan C , Chen N , Shuo Z . High-performance elastic management for cloud containers based on predictive message scheduling. Future Internet, 2017, 9( 4): 87–

[84] Hou X, Liu J, Li C, Guo M. Unleashing the scalability potential of power-constrained data center in the microservice era. In: Proceedings of the 48th International Conference on Parallel Processing. 2019, 10

[85] Guerrero C , Lera I , Juiz C . Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications. The Journal of Supercomputing, 2018, 74( 7): 2956– 2983

[86] Leng X, Juang T H, Chen Y, Liu H. AOMO: an AI-aided optimizer for microservices orchestration. In: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos. 2019, 1−2

[87] Klock S, van der Werf J M E M, Guelen J P, Jansen S. Workload-based clustering of coherent feature sets in microservice architectures. In: Proceedings of 2017 IEEE International Conference on Software Architecture. 2017, 11−20

[88] Monteiro D, Gadelha R, Maia P H M, Rocha L S, Mendonça N C. Beethoven: an event-driven lightweight platform for microservice orchestration. In: Proceedings of the 12th European Conference on Software Architecture. 2018, 191−199

[89] Guo D, Wang W, Zeng G, Wei Z. Microservices architecture based cloudware deployment platform for service computing. In: Proceedings of 2016 IEEE Symposium on Service-Oriented System Engineering. 2016, 358−363

[90] Rufino J, Alam M, Ferreira J, Rehman A, Tsang K F. Orchestration of containerized microservices for IIoT using Docker. In: Proceedings of 2017 IEEE International Conference on Industrial Technology. 2017, 1532−1536

[91] Meulenhoff P J, Ostendorf D R, Živković M, Meeuwissen H B, Gijsen B M M. Intelligent overload control for composite web services. In: Proceedings of the 7th International Joint Conference on Service-Oriented Computing. 2009, 34−49

[92] Welsh M, Culler D, Brewer E. SEDA: an architecture for well-conditioned, scalable internet services. In: Proceedings of the 18th ACM Symposium on Operating Systems Principles. 2001, 230−243

[93] Yang H, Breslow A, Mars J, Tang L. Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers. In: Proceedings of the 40th Annual International Symposium on Computer Architecture. 2013, 607−618

[94] Delimitrou C, Kozyrakis C. Quasar: resource-efficient and QoS-aware cluster management. In: Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems. 2014, 127−144

[95] Hou X, Li C, Liu J, Zhang L, Ren S, Leng J, Chen Q, Guo M. AlphaR: learning-powered resource management for irregular, dynamic microservice graph. In: Proceedings of 2021 IEEE International Parallel and Distributed Processing Symposium. 2021, 797−806

[96] Alipour H, Liu Y. Online machine learning for cloud resource provisioning of microservice backend systems. In: Proceedings of 2017 IEEE International Conference on Big Data. 2017, 2433−2441

[97] Chang M A, Panda A, Tsai Y C, Wang H, Shenker S. ThrottleBot - performance without insight. 2017, arXiv preprint arXiv: 1711.00618

[98] Zhou H, Chen M, Lin Q, Wang Y, She X, Liu S, Gu R, Ooi B C, Yang J. Overload control for scaling WeChat microservices. In: Proceedings of the ACM Symposium on Cloud Computing. 2018, 149−161

[99] Suresh L, Bodik P, Menache I, Canini M, Ciucu F. Distributed resource management across process boundaries. In: Proceedings of 2017 Symposium on Cloud Computing. 2017, 611−623

[100] Xu M , Toosi A N , Buyya R . iBrownout: an integrated approach for managing energy and brownout in container-based clouds. IEEE Transactions on Sustainable Computing, 2019, 4( 1): 53– 66

[101] Hou X, Li C, Liu J, Zhang L, Hu Y, Guo M. ANT-man: towards agile power management in the microservice era. In: Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis. 2020, 78

[102] Chou C H, Bhuyan L N, Wong D. µDPM: dynamic power management for the microsecond era. In: Proceedings of 2019 International Symposium on High Performance Computer Architecture. 2019, 120−132

[103] Mirhosseini A, Sriraman A, Wenisch T F. Enhancing server efficiency in the face of killer microseconds. In: Proceedings of 2019 IEEE International Symposium on High Performance Computer Architecture. 2019, 185−198

[104] Kasture H, Bartolini D B, Beckmann N, Sanchez D. Rubik: fast analytical power management for latency-critical systems. In: Proceedings of the 48th Annual IEEE/ACM International Symposium on Microarchitecture. 2015, 598−610

[105] Lo D, Cheng L, Govindaraju R, Barroso L A, Kozyrakis C. Towards energy proportionality for large-scale latency-critical workloads. In: Proceedings of the 41st ACM/IEEE International Symposium on Computer Architecture. 2014, 301−312

[106] Liu Y, Draper S C, Kim N S. SleepScale: runtime joint speed scaling and sleep states management for power efficient data centers. In: Proceedings of the 41st Annual International Symposium on Computer Architecuture. 2014, 313−324

[107] Boucher S, Kalia A, Andersen D G, Kaminsky M. Putting the “micro” back in microservice. In: Proceedings of 2018 USENIX Annual Technical Conference. 2018, 645−650

[108] Oakes E, Yang L, Zhou D, Houck K, Harter T, Arpaci-Dusseau A, Arpaci-Dusseau R H. SOCK: rapid task provisioning with serverless-optimized containers. In: Proceedings of 2018 USENIX Annual Technical Conference. 2018, 57−69

[109] Akkus I, Chen R, Rimac I, Stein M, Satzke K, Beck A, Aditya P, Hilt V. SAND: towards high-performance serverless computing. In: Proceedings of 2018 USENIX Conference on Usenix Annual Technical Conference. 2018, 923−935

[110] Luo X, Ren F, Zhang T. High performance userspace networking for containerized microservices. In: Proceedings of the 16th International Conference on Service-Oriented Computing. 2018, 57−72

Acknowledgements

This work was sponsored by the National Natural Science Foundation of China (Grant No. 61972247). Corresponding author is Chao Li from Shanghai Jiao Tong University, China.

RIGHTS & PERMISSIONS

2022 Higher Education Press