System log isolation for containers

Abstract

Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility. Isolation is a fundamental property of containers and weak isolation could cause significant performance degradation and security vulnerability. However, existing works have almost not discussed the isolation problems of system log which is critical for monitoring and maintenance of containerized applications. In this paper, we present a detailed isolation analysis of system log in current container environment. First, we find several system log isolation problems which can cause significant impacts on system usability, security, and efficiency. For example, system log accidentally exposes information of host and co-resident containers to one container, causing information leakage. Second, we reveal that the root cause of these isolation problems is that containers share the global log configuration, the same log storage, and the global log view. To address these problems, we design and implement a system named private logs (POGs). POGs provides each container with its own log configuration and stores logs individually for each container, avoiding log configuration and storage sharing, respectively. In addition, POGs enables private log view to help distinguish which container the logs belong to. The experimental results show that POGs can effectively enhance system log isolation for containers with negligible performance overhead.

Keywords

container isolation / system log / cgroup / namespace / cloud computing

Kun Wang received the PhD degree from Huazhong University of Science and Technology (HUST), China in 2023. Currently he is an assistant research fellow at the College of Information and Communication in National University of Defense Technology, China. His current research interests include cloud computing, container virtualization, kernel resource isolation and intelligent operating system

Song Wu received the PhD degree from Huazhong University of Science and Technology (HUST), China in 2003. He is a professor of computer science at HUST. He currently serves as the vice dean of the School of Computer Science and Technology and the vice head of Service Computing Technology and System Lab (SCTS) and the Cluster and Grid Computing Lab (CGCL) in HUST. His current research interests include cloud resource scheduling and system virtualization

Yanxiang Cui received his BS degree from Huazhong University of Science and Technology (HUST), China in 2021 and is currently pursuing his MS degree in Service Computing Technology and System Lab (SCTS) and Cluster and Grid Lab(CGCL) in HUST. His research interests include operating systems, performance evaluation, and lightweight virtualization technologies

Zhuo Huang received the BS from Huazhong Agricultural University (HZAU), China in 2014. Currently he is a PhD candidate student in Service Computing Technology and System Lab (SCTS) and Cluster and Grid Lab (CGCL), Huazhong University of Science and Technology (HUST), China. His current research interests include container virtualization, serverless computing optimization, and storage system

Hao Fan received the PhD degree from Huazhong University of Science and Technology (HUST), China in 2021. Currently he is working as a post-doctor in Service Computing Technology and System Lab (SCTS) and Cluster and Grid Lab (CGCL) in HUST. His current research interests include container technology and storage system

Hai Jin is a Chair Professor of computer science at Huazhong University of Science and Technology (HUST), China. Jin received his PhD degree in computer engineering from HUST, China in 1994. In 1996, he was awarded a German Academic Exchange Service fellowship to visit the Technical University of Chemnitz. Jin worked at The University of Hong Kong, China between 1998 and 2000. He was awarded Excellent Youth Award from the National Natural Science Foundation of China in 2001. Jin is a Fellow of IEEE, Fellow of CCF, and a life member of the ACM. He has co-authored more than 20 books and published over 900 research papers. His research interests include computer architecture, parallel and distributed computing, big data processing, data storage, and system security

References

[1] Gu L, Zeng D, Hu J, Jin H, Guo S, Zomaya A Y. Exploring layered container structure for cost efficient microservice deployment. In: Proceedings of IEEE Conference on Computer Communications. 2021, 1−9

[2] Li Z, Cheng J, Chen Q, Guan E, Bian Z, Tao Y, Zha B, Wang Q, Han W, Guo M. RunD: a lightweight secure container runtime for high-density deployment and high-concurrency startup in serverless computing. In: Proceedings of 2022 USENIX Annual Technical Conference. 2022, 53−68

[3] Suo K, Zhao Y, Chen W, Rao J. An analysis and empirical study of container networks. In: Proceedings of IEEE Conference on Computer Communications. 2018, 189−197

[4] Zeng R, Hou X, Zhang L, Li C, Zheng W, Guo M . Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities. Frontiers of Computer Science, 2022, 16( 6): 166106

[5] Soltesz S, Pötzl H, Fiuczynski M E, Bavier A, Peterson L. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems. 2007, 275−287

[6] Zhuang Z, Tran C, Weng J, Ramachandra H, Sridharan B. Taming memory related performance pitfalls in Linux Cgroups. In: Proceedings of 2017 International Conference on Computing, Networking and Communications. 2017, 531−535

[7] Laadan O, Nieh J. Operating system virtualization: practice and experience. In: Proceedings of the 3rd Annual Haifa Experimental Systems Conference. 2010, 17

[8] Huang Z, Wu S, Jiang S, Jin H. FastBuild: accelerating docker image building for efficient development and deployment of container. In: Proceedings of the 35th Symposium on Mass Storage Systems and Technologies. 2019, 28−37

[9] Yang N, Shen W, Li J, Yang Y, Lu K, Xiao J, Zhou T, Qin C, Yu W, Ma J, Ren K. Demons in the shared kernel: abstract resource attacks against OS-level virtualization. In: Proceedings of 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021, 764−778

[10] Hua Z, Yu Y, Gu J, Xia Y, Chen H, Zang B . TZ-container: protecting container from untrusted OS with ARM TrustZone. Science China Information Sciences, 2021, 64( 9): 192101

[11] Plauth M, Feinbube L, Polze A. A performance survey of lightweight virtualization techniques. In: Proceedings of the 6th IFIP WG 2.14 European Conference on Service-Oriented and Cloud Computing. 2017, 34−48

[12] Matthews J N, Hu W, Hapuarachchi M, Deshane T, Dimatos D, Hamilton G, McCabe M, Owens J. Quantifying the performance isolation properties of virtualization systems. In: Proceedings of 2007 Workshop on Experimental Computer Science. 2007, 6−es

[13] Felter W, Ferreira A, Rajamony R, Rubio J. An updated performance comparison of virtual machines and Linux containers. In: Proceedings of 2015 IEEE International Symposium on Performance Analysis of Systems and Software. 2015, 171−172

[14] Sharma P, Chaufournier L, Shenoy P, Tay Y C. Containers and virtual machines at scale: a comparative study. In: Proceedings of the 17th International Middleware Conference. 2016, 1

[15] Xavier M G, De Oliveira I C, Rossi F D, Dos Passos R D, Matteussi K J, De Rose C A F. A performance isolation analysis of disk-intensive workloads on container-based clouds. In: Proceedings of the 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing. 2015, 253−260

[16] Huang H, Rao J, Wu S, Jin H, Suo K, Wu X. Adaptive resource views for containers. In: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing. 2019, 243−254

[17] Sun Y, Safford D, Zohar M, Pendarakis D, Gu Z, Jaeger T. Security namespace: making Linux security frameworks available to containers. In: Proceedings of the 27th USENIX Security Symposium. 2018, 1423−1439

[18] Gao X, Gu Z, Li Z, Jamjoom H, Wang C. Houdini’s escape: breaking the resource rein of Linux control groups. In: Proceedings of 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019, 1073−1086

[19] Khalid J, Rozner E, Felter W, Xu C, Rajamani K, Ferreira A, Akella A. Iron: isolating network-based CPU in container environments. In: Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation. 2018, 313−328

[20] Li Y, Zhang J, Jiang C, Wan J, Ren Z . PINE: optimizing performance isolation in container environments. IEEE Access, 2019, 7: 30410–30422

[21] Gao X, Gu Z, Kayaalp M, Pendarakis D, Wang H. ContainerLeaks: emerging security threats of information leakages in container clouds. In: Proceedings of the 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 2017, 237−248

[22] Du M, Li F, Zheng G, Srikumar V. DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017, 1285−1298

[23] Love R. Linux Kernel Development. 3rd ed. New York: Pearson Education, 2010

[24] Merkel D . Docker: lightweight Linux containers for consistent development and deployment. Linux Journal, 2014, 2014( 239): 2

[25] Xie X L, Wang P, Wang Q. The performance analysis of Docker and rkt based on Kubernetes. In: Proceedings of the 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery. 2017, 2137−2141

[26] Senthil K S. Practical LXC and LXD: Linux Containers for Virtualization and Orchestration. Berkeley: Apress, 2017

[27] Yang Y, Shen W, Ruan B, Liu W, Ren K. Security challenges in the container cloud. In: Proceedings of the 3rd IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications. 2021, 137−145

[28] Lin X, Lei L, Wang Y, Jing J, Sun K, Zhou Q. A measurement study on Linux container security: attacks and countermeasures. In: Proceedings of the 34th Annual Computer Security Applications Conference. 2018, 418−429

[29] Masti R J, Rai D, Ranganathan A, Müller C, Thiele L, Capkun S, Zürich E. Thermal covert channels on multi-core platforms. In: Proceedings of the 24th USENIX Security Symposium. 2015, 865−880

[30] He S, Lin Q, Lou J G, Zhang H, Lyu M R, Zhang D. Identifying impactful service system problems via log analysis. In: Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2018, 60−70

[31] Lin Q, Zhang H, Lou J G, Zhang Y, Chen X. Log clustering based problem identification for online service systems. In: Proceedings of the 38th IEEE/ACM International Conference on Software Engineering Companion. 2016, 102−111

[32] Wu S, Huang Z, Chen P, Fan H, Ibrahim S, Jin H. Container-aware I/O stack: bridging the gap between container storage drivers and solid state devices. In: Proceedings of the 18th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2022, 18−30

[33] Gu L, Guan J, Wu S, Jin H, Rao J, Suo K, Zeng D. CNTC: a container aware network traffic control framework. In: Proceedings of the 14th International Conference of Green, Pervasive, and Cloud Computing. 2019, 208−222

[34] Shen Z, Sun Z, Sela G E, Bagdasaryan E, Delimitrou C, Van Renesse R, Weatherspoon H. X-containers: breaking down barriers to improve performance and isolation of cloud-native containers. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 121−135

[35] Manco F, Lupu C, Schmidt F, Mendes J, Kuenzer S, Sati S, Yasukata K, Raiciu C, Huici F. My VM is lighter (and safer) than your container. In: Proceedings of the 26th Symposium on Operating Systems Principles. 2017, 218−233

[36] Randazzo A, Tinnirello I. Kata containers: an emerging architecture for enabling MEC services in fast and secure way. In: Proceedings of the 6th International Conference on Internet of Things: Systems, Management and Security. 2019, 209−214

[37] Anjali, Caraza-Harter T, Swift M M. Blending containers and virtual machines: a study of firecracker and gVisor. In: Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2020, 101−113

[38] Beschastnikh I, Brun Y, Schneider S, Sloan M, Ernst M D. Leveraging existing instrumentation to automatically infer invariant-constrained models. In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering. 2011, 267−277

[39] Shang W, Jiang Z M, Hemmati H, Adams B, Hassan A E, Martin P. Assisting developers of big data analytics applications when deploying on hadoop clouds. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 402−411

[40] Ding R, Fu Q, Lou J G, Lin Q, Zhang D, Xie T. Mining historical issue repositories to heal large-scale online service systems. In: Proceedings of the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 2014, 311−322

[41] Rakha M S, Bezemer C P, Hassan A E . Revisiting the performance evaluation of automated approaches for the retrieval of duplicate issue reports. IEEE Transactions on Software Engineering, 2018, 44( 12): 1245–1268

[42] He S, Zhu J, He P, Lyu M R. Experience report: system log analysis for anomaly detection. In: Proceedings of the 27th IEEE International Symposium on Software Reliability Engineering. 2016, 207−218

[43] Lim M H, Lou J G, Zhang H, Fu Q, Teoh A B J, Lin Q, Ding R, Zhang D. Identifying recurrent and unknown performance issues. In: Proceedings of 2014 IEEE International Conference on Data Mining. 2014, 320−329

Acknowledgements

We thank the anonymous reviewers for their helpful feedback. This work was supported by the National Key R&D Program (2022YFB4500704), the National Natural Science Foundation of China (Grant No. 62032008).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2024 The Author(s) 2024. This article is published with open access at link.springer.com and journal.hep.com.cn