System log isolation for containers

Abstract

Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility. Isolation is a fundamental property of containers and weak isolation could cause significant performance degradation and security vulnerability. However, existing works have almost not discussed the isolation problems of system log which is critical for monitoring and maintenance of containerized applications. In this paper, we present a detailed isolation analysis of system log in current container environment. First, we find several system log isolation problems which can cause significant impacts on system usability, security, and efficiency. For example, system log accidentally exposes information of host and co-resident containers to one container, causing information leakage. Second, we reveal that the root cause of these isolation problems is that containers share the global log configuration, the same log storage, and the global log view. To address these problems, we design and implement a system named private logs (POGs). POGs provides each container with its own log configuration and stores logs individually for each container, avoiding log configuration and storage sharing, respectively. In addition, POGs enables private log view to help distinguish which container the logs belong to. The experimental results show that POGs can effectively enhance system log isolation for containers with negligible performance overhead.

Graphical abstract

Keywords

container isolation / system log / cgroup / namespace / cloud computing

References

[1] Gu L, Zeng D, Hu J, Jin H, Guo S, Zomaya A Y. Exploring layered container structure for cost efficient microservice deployment. In: Proceedings of IEEE Conference on Computer Communications. 2021, 1−9

[2] Li Z, Cheng J, Chen Q, Guan E, Bian Z, Tao Y, Zha B, Wang Q, Han W, Guo M. RunD: a lightweight secure container runtime for high-density deployment and high-concurrency startup in serverless computing. In: Proceedings of 2022 USENIX Annual Technical Conference. 2022, 53−68

[3] Suo K, Zhao Y, Chen W, Rao J. An analysis and empirical study of container networks. In: Proceedings of IEEE Conference on Computer Communications. 2018, 189−197

[4] Zeng R, Hou X, Zhang L, Li C, Zheng W, Guo M . Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities. Frontiers of Computer Science, 2022, 16( 6): 166106

[5] Soltesz S, Pötzl H, Fiuczynski M E, Bavier A, Peterson L. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems. 2007, 275−287

[6] Zhuang Z, Tran C, Weng J, Ramachandra H, Sridharan B. Taming memory related performance pitfalls in Linux Cgroups. In: Proceedings of 2017 International Conference on Computing, Networking and Communications. 2017, 531−535

[7] Laadan O, Nieh J. Operating system virtualization: practice and experience. In: Proceedings of the 3rd Annual Haifa Experimental Systems Conference. 2010, 17

[8] Huang Z, Wu S, Jiang S, Jin H. FastBuild: accelerating docker image building for efficient development and deployment of container. In: Proceedings of the 35th Symposium on Mass Storage Systems and Technologies. 2019, 28−37

[9] Yang N, Shen W, Li J, Yang Y, Lu K, Xiao J, Zhou T, Qin C, Yu W, Ma J, Ren K. Demons in the shared kernel: abstract resource attacks against OS-level virtualization. In: Proceedings of 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021, 764−778

[10] Hua Z, Yu Y, Gu J, Xia Y, Chen H, Zang B . TZ-container: protecting container from untrusted OS with ARM TrustZone. Science China Information Sciences, 2021, 64( 9): 192101

[11] Plauth M, Feinbube L, Polze A. A performance survey of lightweight virtualization techniques. In: Proceedings of the 6th IFIP WG 2.14 European Conference on Service-Oriented and Cloud Computing. 2017, 34−48

[12] Matthews J N, Hu W, Hapuarachchi M, Deshane T, Dimatos D, Hamilton G, McCabe M, Owens J. Quantifying the performance isolation properties of virtualization systems. In: Proceedings of 2007 Workshop on Experimental Computer Science. 2007, 6−es

[13] Felter W, Ferreira A, Rajamony R, Rubio J. An updated performance comparison of virtual machines and Linux containers. In: Proceedings of 2015 IEEE International Symposium on Performance Analysis of Systems and Software. 2015, 171−172

[14] Sharma P, Chaufournier L, Shenoy P, Tay Y C. Containers and virtual machines at scale: a comparative study. In: Proceedings of the 17th International Middleware Conference. 2016, 1

[15] Xavier M G, De Oliveira I C, Rossi F D, Dos Passos R D, Matteussi K J, De Rose C A F. A performance isolation analysis of disk-intensive workloads on container-based clouds. In: Proceedings of the 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing. 2015, 253−260

[16] Huang H, Rao J, Wu S, Jin H, Suo K, Wu X. Adaptive resource views for containers. In: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing. 2019, 243−254

[17] Sun Y, Safford D, Zohar M, Pendarakis D, Gu Z, Jaeger T. Security namespace: making Linux security frameworks available to containers. In: Proceedings of the 27th USENIX Security Symposium. 2018, 1423−1439

[18] Gao X, Gu Z, Li Z, Jamjoom H, Wang C. Houdini’s escape: breaking the resource rein of Linux control groups. In: Proceedings of 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019, 1073−1086

[19] Khalid J, Rozner E, Felter W, Xu C, Rajamani K, Ferreira A, Akella A. Iron: isolating network-based CPU in container environments. In: Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation. 2018, 313−328

[20] Li Y, Zhang J, Jiang C, Wan J, Ren Z . PINE: optimizing performance isolation in container environments. IEEE Access, 2019, 7: 30410–30422

[21] Gao X, Gu Z, Kayaalp M, Pendarakis D, Wang H. ContainerLeaks: emerging security threats of information leakages in container clouds. In: Proceedings of the 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 2017, 237−248

[22] Du M, Li F, Zheng G, Srikumar V. DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017, 1285−1298

[23] Love R. Linux Kernel Development. 3rd ed. New York: Pearson Education, 2010

[24] Merkel D . Docker: lightweight Linux containers for consistent development and deployment. Linux Journal, 2014, 2014( 239): 2

[25] Xie X L, Wang P, Wang Q. The performance analysis of Docker and rkt based on Kubernetes. In: Proceedings of the 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery. 2017, 2137−2141

[26] Senthil K S. Practical LXC and LXD: Linux Containers for Virtualization and Orchestration. Berkeley: Apress, 2017

[27] Yang Y, Shen W, Ruan B, Liu W, Ren K. Security challenges in the container cloud. In: Proceedings of the 3rd IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications. 2021, 137−145

[28] Lin X, Lei L, Wang Y, Jing J, Sun K, Zhou Q. A measurement study on Linux container security: attacks and countermeasures. In: Proceedings of the 34th Annual Computer Security Applications Conference. 2018, 418−429

[29] Masti R J, Rai D, Ranganathan A, Müller C, Thiele L, Capkun S, Zürich E. Thermal covert channels on multi-core platforms. In: Proceedings of the 24th USENIX Security Symposium. 2015, 865−880

[30] He S, Lin Q, Lou J G, Zhang H, Lyu M R, Zhang D. Identifying impactful service system problems via log analysis. In: Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2018, 60−70

[31] Lin Q, Zhang H, Lou J G, Zhang Y, Chen X. Log clustering based problem identification for online service systems. In: Proceedings of the 38th IEEE/ACM International Conference on Software Engineering Companion. 2016, 102−111

[32] Wu S, Huang Z, Chen P, Fan H, Ibrahim S, Jin H. Container-aware I/O stack: bridging the gap between container storage drivers and solid state devices. In: Proceedings of the 18th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2022, 18−30

[33] Gu L, Guan J, Wu S, Jin H, Rao J, Suo K, Zeng D. CNTC: a container aware network traffic control framework. In: Proceedings of the 14th International Conference of Green, Pervasive, and Cloud Computing. 2019, 208−222

[34] Shen Z, Sun Z, Sela G E, Bagdasaryan E, Delimitrou C, Van Renesse R, Weatherspoon H. X-containers: breaking down barriers to improve performance and isolation of cloud-native containers. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 121−135

[35] Manco F, Lupu C, Schmidt F, Mendes J, Kuenzer S, Sati S, Yasukata K, Raiciu C, Huici F. My VM is lighter (and safer) than your container. In: Proceedings of the 26th Symposium on Operating Systems Principles. 2017, 218−233

[36] Randazzo A, Tinnirello I. Kata containers: an emerging architecture for enabling MEC services in fast and secure way. In: Proceedings of the 6th International Conference on Internet of Things: Systems, Management and Security. 2019, 209−214

[37] Anjali, Caraza-Harter T, Swift M M. Blending containers and virtual machines: a study of firecracker and gVisor. In: Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2020, 101−113

[38] Beschastnikh I, Brun Y, Schneider S, Sloan M, Ernst M D. Leveraging existing instrumentation to automatically infer invariant-constrained models. In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering. 2011, 267−277

[39] Shang W, Jiang Z M, Hemmati H, Adams B, Hassan A E, Martin P. Assisting developers of big data analytics applications when deploying on hadoop clouds. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 402−411

[40] Ding R, Fu Q, Lou J G, Lin Q, Zhang D, Xie T. Mining historical issue repositories to heal large-scale online service systems. In: Proceedings of the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. 2014, 311−322

[41] Rakha M S, Bezemer C P, Hassan A E . Revisiting the performance evaluation of automated approaches for the retrieval of duplicate issue reports. IEEE Transactions on Software Engineering, 2018, 44( 12): 1245–1268

[42] He S, Zhu J, He P, Lyu M R. Experience report: system log analysis for anomaly detection. In: Proceedings of the 27th IEEE International Symposium on Software Reliability Engineering. 2016, 207−218

[43] Lim M H, Lou J G, Zhang H, Fu Q, Teoh A B J, Lin Q, Ding R, Zhang D. Identifying recurrent and unknown performance issues. In: Proceedings of 2014 IEEE International Conference on Data Mining. 2014, 320−329

RIGHTS & PERMISSIONS

The Author(s) 2024. This article is published with open access at link.springer.com and journal.hep.com.cn