Kubernetes application performance benchmarking on heterogeneous CPU architecture: An experimental review

Abstract

With the rapid advancement of cloud technologies, cloud services have enormously contributed to the cloud community for application development life-cycle. In this context, Kubernetes has played a pivotal role as a cloud computing tool, enabling developers to adopt efficient and automated deployment strategies. Using Kubernetes as an orchestration tool and a cloud computing system as a manager of the infrastructures, developers can boost the development and deployment process. With cloud providers such as GCP, AWS, Azure, and Oracle offering Kubernetes services, the availability of both x86 and ARM platforms has become evident. However, while x86 currently dominates the market, ARM-based solutions have seen limited adoption, with only a few individuals actively working on ARM deployments. This study explores the efficiency and cost-effectiveness of implementing Kubernetes on different CPU platforms. By comparing the performance of x86 and ARM platforms, this research seeks to ascertain whether transitioning to ARM presents a more advantageous option for Kubernetes deployments. Through a comprehensive evaluation of scalability, cost, and overall performance, this study aims to shed light on the viability of leveraging ARM on different CPUs by providing valuable insights.

Keywords

Kubernetes / K8s / K3s / Serverless computing / Container as a Service (CaaS) / Container orchestration / Docker / CPU architecture / x86 / ARM

References

[1] J.M. Parra-Ullauri, H. Madhukumar, A.-C. Nicolaescu, X. Zhang, A. Bravalheri, R. Hussain, X. Vasilakos, R. Nejabati, D. Simeonidou, kubeFlower: A privacy-preserving framework for Kubernetes-based federated learning in cloud-edge environments, Future Gener. Comput. Syst. 157 (2024) 558-572, http://dx.doi.org/10.1016/j.future.2024.03.041.

[2] J. Noor, S.I. Salim, A.A.A. Islam, Strategizing secured image storing and efficient image retrieval through a new cloud framework, J. Netw. Comput. Appl. 192 (2021) 103167, http://dx.doi.org/10.1016/j.jnca.2021.103167.

[3] J. Noor, M.N.H. Shanto, J.J. Mondal, M.G. Hossain, S. Chellappan, A. B.M.A. Al Islam, Orchestrating image retrieval and storage over a cloud system, IEEE Trans. Cloud Comput. 11 (2) (2023) 1794-1806, http://dx.doi.org/10.1109/tcc.2022.3162790, URL http://dx.doi.org/10.1109/TCC.2022.3162790

[4] J. Noor, H.I. Akbar, R.A. Sujon, A.A. Al Islam,Secure processing-aware media storage (SPMS), in: 2017 IEEE 36th International Performance Computing and Communications Conference, IPCCC, IEEE, 2017, http://dx.doi.org/10.1109/pccc.2017.8280457.

[5] H. Liang, Z. Zhang, C. Hu, Y. Gong, D. Cheng, A survey on spatio-temporal big data analytics ecosystem: Resource management, processing platform, and applications, IEEE Trans. Big Data 10 (2) (2024) 174-193, http://dx.doi.org/10.1109/tbdata.2023.3342619.

[6] S. Nasrin, T.F. Sahryer, A.B. M.A.A. Islam, J. Noor,Feature and performance based comparative study on serverless frameworks, in: 2021 24th International Conference on Computer and Information Technology (ICCIT), IEEE, 2021, pp. 1-6, http://dx.doi.org/10.1109/iccit54785.2021.9689779.

[7] Pavithra, Google cloud interview questions, 2022, URL https://laraonlinetraining.com/google-cloud-interview-questions/.

[8] C. Carrión, Kubernetes as a standard container orchestrator - A bibliometric analysis, J. Grid Comput. 20 (4) (2022) 42, http://dx.doi.org/10.1007/s10723-022-09629-8.

[9] Pods | Official Documentation of Kubernetes, 2023, URL https://kubernetes.io/docs/concepts/workloads/pods.

[10] T. Menouer, KCSS: Kubernetes container scheduling strategy, J. Supercomput. 77 (5) (2020) 4267-4293, http://dx.doi.org/10.1007/s11227-020-03427-3.

[11] D. Bernstein, Containers and cloud: From LXC to Docker to Kubernetes, IEEE Cloud Computing 1 (3) (2014) 81-84, http://dx.doi.org/10.1109/mcc.2014.51.

[12] C. Boettiger, An introduction to docker for reproducible research, ACM SIGOPS Oper. Syst. Rev. 49 (1) (2015) 71-79, http://dx.doi.org/10.1145/2723872.2723882.

[13] M.H. Todorov,Design and deployment of Kubernetes Cluster on Raspberry Pi OS, in: 2021 29th National Conference with International Partici-pation (TELECOM), IEEE, 2021, pp. 104-107, http://dx.doi.org/10.1109/telecom53156.2021.9659651.

[14] S. Telenyk, O. Sopov, E. Zharikov, G. Nowakowski, A comparison of Kubernetes and Kubernetes-compatible platforms, in: 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), 1, IEEE, 2021, pp. 313-317, http://dx.doi.org/10.1109/idaacs53288.2021.9660392.

[15] K. Gupta, T. Sharma,Changing trends in computer architecture : A comprehensive analysis of ARM and x86 processors, Int. J. Sci. Res. Comput. Sci Eng. Inf. Technol. (2021) 619-631, http://dx.doi.org/10.32628/cseit2173188.

[16] H. Shafiei, A. Khonsari, P. Mousavi, Serverless computing: A survey of opportunities, challenges, and applications, ACM Comput. Surv. 54 (11s) (2022) 1-32, http://dx.doi.org/10.1145/3510611.

[17] J. Shah, D. Dubaria,Building modern clouds: using docker, kubernetes & Google cloud platform, in: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC, IEEE, 2019, pp. 0184-0189, http://dx.doi.org/10.1109/ccwc.2019.8666479.

[18] V. Tamizhkumaran, MULTI-cloud: What is it and what are the ways to make best use of it? 2022, URL https://mobiritz.com/technology/multi-cloud-what-is-it-and-what-are-the-ways-to-make-best-use-of-it/.

[19] M.A. Kamal, H.W. Raza, M.M. Alam, M.M. Su’ud, Highlight the features of AWS, GCP and Microsoft Azure that have an impact when choosing a cloud service provider, Int. J. Recent Technol. Eng. ( IJRTE) 8 (5) (2020) 4124-4132, http://dx.doi.org/10.35940/ijrte.d8573.018520.

[20] M. Deb, A. Choudhury, Hybrid cloud: A new paradigm in cloud computing, Mach. Learn. Tech. Anal. Cloud Secur. (2021) 1-23, http://dx.doi.org/10.1002/9781119764113.ch1.

[21] M. Kyryk, N. Pleskanka, M. Pleskanka, V. Kyryk, Infrastructure as code and microservices for intent-based cloud networking, in: Future Intent-Based Networking: On the QoS Robust and Energy Efficient Heterogeneous Software Defined Networks, Springer, 2021, pp. 51-68, http://dx.doi.org/10.1007/978-3-030-92435-5_4.

[22] K. Djemame, Serverless computing: Introduction and research challenges,in: International Conference on the Economics of Grids, Clouds, Systems, and Services, Springer Nature, 2022, pp. 15-23.

[23] S.K. Mondal, R. Pan, H.D. Kabir, T. Tian, H.-N. Dai, Kubernetes in IT administration and serverless computing: An empirical study and research challenges, J. Supercomput. 78 (2) (2022) 1-51, http://dx.doi.org/10.1007/s11227-021-03982-3.

[24] A. Png, H. Helskyaho, Exposing functionality with API gateway, in: Extending Oracle Application Express with Oracle Cloud Features: A Guide To Enhancing APEX Web Applications with Cloud-Native and Machine Learning Technologies, Springer, 2022, pp. 111-138, http://dx.doi.org/10.1007/978-1-4842-8170-3_4.

[25] A. Engelsrud, Managing PeopleSoft on the Oracle Cloud, A Press, 2019, http://dx.doi.org/10.1007/978-1-4842-4546-0.

[26] P.G. López, A. Arjona, J. Sampé, A. Slominski, L. Villard, Triggerflow: triggerbased orchestration of serverless workflows, in: Proceedings of the 14th ACM International Conference on Distributed and Event-Based Systems, 2020, pp. 3-14, http://dx.doi.org/10.1145/3401025.3401731.

[27] P. Raj, S. Vanga, A. Chaudhary, Setting up a Kubernetes Cluster using Azure Kubernetes Service, Wiley-IEEE Press (2023) http://dx.doi.org/10.1002/9781119814795.ch11.

[28] T. Melissaris, K. Nabar, R. Radut, S. Rehmtulla, A. Shi, S. Chandrashekar, I. Papapanagiotou,Elastic cloud services: scaling snowflake’s control plane, in: Proceedings of the 13th Symposium on Cloud Computing, ACM, 2022, pp. 142-157, http://dx.doi.org/10.1145/3542929.3563483.

[29] E. Blem, J. Menon, K. Sankaralingam,A detailed analysis of contemporary arm and x86 architectures, 2013, URL https://minds.wisconsin.edu/handle/1793/64923.

[30] D.C. Schuurman, Step-by-step design and simulation of a simple CPU architecture, in: Proceeding of the 44th ACM Technical Symposium on Computer Science Education, 2013, pp. 335-340, http://dx.doi.org/10.1145/2445196.2445296.

[31] J. Phillips, Simulation of a simple CPU design and its use as an instructional tool in a computer organization course, J. Comput. Sci. Coll. 22 (6) (2007) 140-146.

[32] A. Sodan, J. Machina, A. Deshmeh, K. Macnaughton, B. Esbaugh, Parallelism via multithreaded and multicore CPUs, Computer 43 (3) (2010) 24-32, http://dx.doi.org/10.1109/mc.2010.75.

[33] J. Hopkins, What is an ARM processor? comparison to x86 and its advantages and disadvantages, 2022, URL https://www.totalphase.com/blog/2022/03/what-is-arm-processor-comparison-x86-and-advantages-disadvantages.

[34] S. Jain, Advantages and disadvantages of ARM processor, 2020, URL https://www.geeksforgeeks.org/advantages-and-disadvantages-of-arm-processor.

[35] K. tej Koganti, E. Patnala, S.S. Narasingu, J. Chaitanya, Virtualization technology in cloud computing environment, Int. J. Emerg. Technol. Adv. Eng. 3 (3) (2013).

[36] J. Turnbull, The Docker Book: Containerization is the new virtualization, James Turnbull, 2014.

[37] C. Pahl, Containerization and the PaaS Cloud, IEEE Cloud Comput. 2 (3) (2015) 24-31, http://dx.doi.org/10.1109/mcc.2015.51.

[38] D. Reis, B. Piedade, F.F. Correia, J.P. Dias, A. Aguiar, Developing docker and docker-compose specifications: A developers’ survey, IEEE Access 10 (2021) 2318-2329, http://dx.doi.org/10.1109/access.2021.3137671.

[39] T. Bui, Analysis of Docker Security, 2015, http://dx.doi.org/10.48550/ARXIV.1501.02967.

[40] B. Russell, Passive benchmarking with docker LXC, 2014, URL https://www.slideshare.net/BodenRussell/kvm-and-docker-lxc-benchmarking-with-openstack.

[41] A.P. Ferreira, R. Sinnott,A performance evaluation of containers running on managed kubernetes services, in: 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), IEEE, 2019, pp. 199-208, http://dx.doi.org/10.1109/cloudcom.2019.00038.

[42] L.P. Dewi, A. Noertjahyana, H.N. Palit, K. Yedutun,Server scalability using kubernetes, in: 2019 4th Technology Innovation Management and Engineering Science International Conference (TIMES-ICON), IEEE, 2019, pp. 1-4, http://dx.doi.org/10.1109/times-icon47539.2019.9024501.

[43] V. Medel, R. Tolosana-Calasanz, J.Á. Bañares, U. Arronategui, O.F. Rana, Characterising resource management performance in Kubernetes, Comput. Electr. Eng. 68 (2018) 286-297, http://dx.doi.org/10.1016/j.compeleceng.2018.03.041.

[44] G. Budigiri, C. Baumann, J.T. Muhlberg, E. Truyen, W. Joosen,Network policies in Kubernetes: Performance evaluation and security analysis, in: 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), IEEE, 2021, pp. 407-412, http://dx.doi.org/10.1109/eucnc/6gsummit51104.2021.9482526.

[45] L. Mercl, J. Pavlik, Public cloud Kubernetes storage performance analysis, in: Computational Collective Intelligence, Springer International Publishing, 2019, pp. 649-660, http://dx.doi.org/10.1007/978-3-030-28374-2_56.

[46] Z. He,Novel container cloud elastic scaling strategy based on Kubernetes, in: 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), IEEE, 2020, pp. 1400-1404, http://dx.doi.org/10.1109/itoec49072.2020.9141552.

[47] S. Kenlon,5 reasons to run Kubernetes on your Raspberry Pi home-lab, 2020, URL https://opensource.com/article/20/8/kubernetes-raspberry-pi, (Accessed on 29 May 2023).

[48] T. Goethals, F.D. Turck, B. Volckaert, Extending kubernetes clusters to low-resource edge devices using virtual kubelets, IEEE Trans. Cloud Comput. 10 (4) (2022) 2623-2636, http://dx.doi.org/10.1109/tcc.2020.3033807.

[49] E. Kristiani, C.-T. Yang, C.-Y. Huang, Y.-T. Wang, P.-C. Ko, The implementation of a cloud-edge computing architecture using OpenStack and kubernetes for air quality monitoring application, Mob. Netw. Appl. 26 (3) (2020) 1070-1092, http://dx.doi.org/10.1007/s11036-020-01620-5.

[50] J. Laukemann, J. Hammer, G. Hager, G. Wellein,Automatic throughput and critical path analysis of x86 and ARM assembly kernels, in: 2019 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS), IEEE, 2019, pp. 1-6, http://dx.doi.org/10.1109/pmbs49563.2019.00006.

[51] V. Medel, O. Rana, J.Á. Bañares, U. Arronategui, Modelling performance & resource management in kubernetes, in:Proceedings of the 9th International Conference on Utility and Cloud Computing, 2016, pp. 257-262.

[52] J. Goodacre, A.N. Sloss, Parallelism and the ARM instruction set architecture, Computer 38 (7) (2005) 42-50, http://dx.doi.org/10.1109/mc.2005.239.

[53] R.V. Aroca, L.M.G. Gonçalves, Towards green data centers: A comparison of x86 and ARM architectures power efficiency, J. Parallel Distrib. Comput. 72 (12) (2012) 1770-1780, http://dx.doi.org/10.1016/j.jpdc.2012.08.005.

[54] Y. Bouizem, N. Parlavantzas, D. Dib, C. Morin,Active-standby for high-availability in faas, in: Proceedings of the 2020 Sixth International Workshop on Serverless Computing, ACM, 2020, pp. 31-36, http://dx.doi.org/10.1145/3429880.3430097.

[55] P. Czarnul, Benchmarking performance of a hybrid intel xeon/xeon phi system for parallel computation of similarity measures between large vectors, Int. J. Parallel Program. 45 (5) (2016) 1091-1107, http://dx.doi.org/10.1007/s10766-016-0455-0.

[56] Docker Overview, 2022, URL https://docs.docker.com/get-started/overview/.

[57] N. Kratzke, Cloud computing costs and benefits: An IT management point of view,in:Service Science: Research and Innovations in the Service Economy, Springer New York, 2012, pp. 185-203, http://dx.doi.org/10.1007/978-1-4614-2326-3_10.

[58] B. Linzel, E. Zhu, G. Flores, J. Liu, S. Dikaleh, How can OpenShift accelerate your kubernetes adoption: A workshop exploring OpenShift features,in:Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering, CASCON ’19, IBM Corp., USA, 2019, pp. 380-381.

[59] H. Fathoni, C.-T. Yang, C.-H. Chang, C.-Y. Huang, Performance comparison of lightweight kubernetes in edge devices, in: Pervasive Systems, Algorithms and Networks, Springer International Publishing, 2019, pp. 304-309, http://dx.doi.org/10.1007/978-3-030-30143-9_25.

[60] B. Blieninger, A. Dietz, U. Baumgarten, Mark8s-A management approach for automotive real-time kubernetes containers in the mobile edge cloud, RAGE 2022 ( 2022) 10.

[61] R. Muddinagiri, S. Ambavane, S. Bayas,Self-hosted Kubernetes: deploying Docker containers locally with Minikube, in: 2019 International Conference on Innovative Trends and Advances in Engineering and Technology, ICITAET, IEEE, 2019, pp. 239-243, http://dx.doi.org/10.1109/icitaet47105.2019.9170208.

[62] C. Pahl, P. Jamshidi, Microservices: A systematic mapping study,in:Proceedings of the 6th International Conference on Cloud Computing and Services Science, SCITEPRESS - Science and Technology Publications, 2016, pp. 137-149, http://dx.doi.org/10.5220/0005785501370146.

[63] G.G. Magalhaes, A.L. Sartor, A.F. Lorenzon, P.O.A. Navaux, A.C. Schneider Beck,How programming languages and paradigms affect performance and energy in multithreaded applications, in: 2016 VI Brazilian Symposium on Computing Systems Engineering, SBESC, IEEE, 2016, http://dx.doi.org/10.1109/sbesc.2016.019.

[64] S. Taherizadeh, M. Grobelnik, Key influencing factors of the kubernetes auto-scaler for computing-intensive microservice-native cloud-based applications, Adv. Eng. Softw. 140 (2020) 102734, http://dx.doi.org/10.1016/j.advengsoft.2019.102734.

[65] P. Jogalekar, M. Woodside, Evaluating the scalability of distributed systems, IEEE Trans. Parallel Distrib. Syst. 11 (6) (2000) 589-603, http://dx.doi.org/10.1109/71.862209, URL http://dx.doi.org/10.1109/71.862209

[66] T. Tanadechopon, B. Kasemsontitum,Performance evaluation of programming languages as API services for cloud environments: A comparative study of PHP, python, node.js and golang, in: 2023 7th International Conference on Information Technology (InCIT), IEEE, 2023, http://dx.doi.org/10.1109/incit60207.2023.10413079.

[67] G. Sayfan, Mastering Kubernetes, Packt Publishing, Birmingham, England, 2023.

[68] L. Larsson, W. Tärneberg, C. Klein, E. Elmroth, M. Kihl, Impact of etcd deployment on Kubernetes, Istio, and application performance, Softw. - Pract. Exp. 50 (10) (2020) 1986-2007, http://dx.doi.org/10.1002/spe.2885.

[69] D. Vohra, S.O. Service, Kubernetes Management Design Patterns:With Docker, CoreOS Linux, and Other Platforms, A Press, Berkeley, Ca, 2017.