CZone: memory-efficient serverless container management with MCPV model

Abstract

Serverless computing usually employs secure containers, which are encapsulated within lightweight microVMs, to isolate function invocations across different tenants. For high security guarantees, such Single-Container-Per-VM (SCPV) model results in large memory waste, as each microVM includes a guestOS, even though they are identical. For memory efficiency of the secure container architecture, we advocate for the dynamic Multi-Containers-Per-VM (MCPV) model. Adopting the MCPV model, a microVM can accommodate multiple containers for the same functions, and the microVM’s memory space adjusts dynamically based on workload.

However, implementing this model necessitates efficient memory hot-plug and hot-unplug techniques. Existing methods either significantly impair function performance within containers or fail to adequately unplug all required pages. To address this challenge, we propose CZone, a dedicated memory hot-plug and hot-unplug design specifically tailored to support the dynamic MCPV model. CZone ensures that the used memory pages of a container are located in contiguous physical memory regions, with each region exclusively allocated to a single container. Experimental results demonstrate that MCPV with CZone brings an 81.81% reduction in startup latency and an 89.87% reduction in memory footprint when compared to the existing SCPV model that utilizes microVM templating, with negligible system performance impact.

Graphical abstract

Keywords

serverless / function-as-a-service / secure container runtime / memory hot-plug

References

[1] Jonas E, Schleier-Smith J, Sreekanti V, Tsai C, Khandelwal A, Pu Q, Shankar V, Carreira J, Krauth K, Yadwadkar N J, Gonzalez J E, Popa R A, Stoica I, Patterson D A. Cloud programming simplified: a berkeley view on serverless computing. 2019, arXiv preprint arXiv: 1902.03383

[2] Castro P, Ishakian V, Muthusamy V, Slominski A. The rise of serverless computing. Communications of the ACM, 2019, 62( 12): 44–54

[3] Li Z, Liu Y, Guo L, Chen Q, Cheng J, Zheng W, Guo M. FaaSFlow: enable efficient workflow execution for function-as-a-service. In: Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2022, 782−796

[4] Buyya R, Srirama S N, Casale G, Calheiros R N, Simmhan Y, Varghese B, Gelenbe E, Javadi B, Vaquero L M, Netto M A S, Toosi A N, Rodriguez M A, Llorente I M, De Capitani De Vimercati S, Samarati P, Milojicic D, Varela C, Bahsoon R, de Assuncao M D, Rana O, Zhou W, Jin H, Gentzsch W, Zomaya A Y, Shen H. A manifesto for future generation cloud computing: research directions for the next decade. ACM Computing Surveys (CSUR), 2019, 51( 5): 105

[5] Shahrad M, Balkind J, Wentzlaff D. Architectural implications of function-as-a-service computing. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture. 2019, 1063−1075

[6] Wang L, Li M, Zhang Y, Ristenpart T, Swift M. Peeking behind the curtains of serverless platforms. In: Proceedings of 2018 USENIX Conference on Usenix Annual Technical Conference. 2018, 133−146

[7] Hendrickson S, Sturdevant S, Harter T, Venkataramani V, Arpaci-Dusseau A C, Arpaci-Dusseau R H. Serverless computation with openLambda. In: Proceedings of the 8th USENIX Conference on Hot Topics in Cloud Computing. 2016, 33−39

[8] AWS Lambda. See aws.amazon.com/lambda/ website, 2024

[9] Google Cloud Functions. See cloud.google.com/functions website, 2024

[10] Microsoft Azure Functions. See azure.microsoft.com/en-us/services/functions website, 2024

[11] Alibaba Function Compute. See alibabacloud.com/product/function-compute website, 2024

[12] Oakes E, Yang L, Zhou D, Houck K, Harter T, Arpaci-Dusseau A C, Arpaci-Dusseau R H. SOCK: rapid task provisioning with serverless-optimized containers. In: Proceedings of 2018 USENIX Conference on Usenix Annual Technical Conference. 2018, 57−70

[13] Kata Containers. See katacontainers.io/ website, 2024

[14] Li Z, Cheng J, Chen Q, Guan E, Bian Z, Tao Y, Zha B, Wang Q, Han W, Guo M. RunD: a lightweight secure container runtime for high-density deployment and high-concurrency startup in serverless computing. In: Proceedings of 2022 USENIX Annual Technical Conference. 2022, 53−68

[15] Agache A, Brooker M, Iordache A, Liguori A, Neugebauer R, Piwonka P, Popa D M. Firecracker: lightweight virtualization for serverless applications. In: Proceedings of the 17th Usenix Conference on Networked Systems Design and Implementation. 2020, 419−434

[16] gVisor: protecting GKE and serverless users in the real world. See cloud.google.com/blog/products/containers-kubernetes/how-gvisor-protects-google-cloud-services-from-cve-2020-14386 website, 2024

[17] Engler D R, Kaashoek M F, O’Toole Jr J. Exokernel: an operating system architecture for application-level resource management. In: Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles. 1995, 251−266

[18] Ustiugov D, Petrov P, Kogias M, Bugnion E, Grot B. Benchmarking, analysis, and optimization of serverless function snapshots. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021, 559−572

[19] Shahrad M, Fonseca R, Goiri I, Chaudhry G I, Batum P, Cooke J, Laureano E, Tresness C, Russinovich M, Bianchini R. Serverless in the wild: characterizing and optimizing the serverless workload at a large cloud provider. In: Proceedings of 2020 USENIX Annual Technical Conference. 2020, 205−218

[20] Azure Functions Traces. See github.com/Azure/AzurePublicDataset website, 2024

[21] Li Z, Guo L, Cheng J, Chen Q, He B, Guo M. The serverless computing survey: a technical primer for design architecture. ACM Computing Surveys (CSUR), 2022, 54( 10s): 220

[22] Lambda: the state of serverless. See www.datadoghq.com/state-of-serverless-2020/ website, 2024

[23] Docker. See www.docker.com/ website, 2024

[24] Linux Container Runtime. See linuxcontainers.org/ website, 2024

[25] Barlev S, Basil Z, Kohanim S, Peleg R, Regev S, Shulman-Peleg A. Secure yet usable: protecting servers and Linux containers. IBM Journal of Research and Development, 2016, 60(4): 12:1−12:10

[26] Mattetti M, Shulman-Peleg A, Allouche Y, Corradi A, Dolev S, Foschini L. Securing the infrastructure and the workloads of Linux containers. In: Proceedings of 2015 IEEE Conference on Communications and Network Security. 2015, 559−567

[27] Akkus I E, Chen R, Rimac I, Stein M, Satzke K, Beck A, Aditya P, Hilt V. SAND: towards high-performance serverless computing. In: Proceedings of the 2018 USENIX Annual Technical Conference. 2018, 923−935

[28] Jia Z, Witchel E. Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021, 152−166

[29] VirtIO Memory Ballooning. See pmhahn.github.io/virtio-balloon/ website, 2024

[30] Hildenbrand D, Schulz M. Virtio-mem: paravirtualized memory hot(un)plug. In: Proceedings of the 17th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2021, 1−14

[31] Introduction to ACPI based memory hot-plug. See events.static.linuxfound.org/sites/events/files/lcjp13_chen.pdf website, 2024

[32] Yu T, Liu Q, Du D, Xia Y, Zang B, Lu Z, Yang P, Qin C, Chen H. Characterizing serverless platforms with serverlessbench. In: Proceedings of the 11th ACM Symposium on Cloud Computing. 2020, 30−44

[33] Memory bandwidth benchmark. See github.com/raas/mbw website, 2024

[34] CORAL benchmark codes. See asc.llnl.gov/coral-benchmarks website, 2024

[35] Lee H, Satyam K, Fox G C. Evaluation of production serverless computing environments. In: Proceedings of the IEEE 11th International Conference on Cloud Computing. 2018, 442−450

[36] Wang Z, Zhao K, Li P, Jacob A, Kozuch M, Mowry T, Skarlatos D. Memento: architectural support for ephemeral memory management in serverless environments. In: Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture. 2023, 122−136

[37] Li H, Li J, Kaufmann A. SimBricks: end-to-end network system evaluation with modular simulation. In: Proceedings of the ACM SIGCOMM 2022 Conference. 2022, 380−396

[38] Mampage A, Karunasekera S, Buyya R. A holistic view on resource management in serverless computing environments: taxonomy and future directions. ACM Computing Surveys (CSUR), 2022, 54( 11s): 222

[39] Suresh A, Somashekar G, Varadarajan A, Kakarla V R, Upadhyay H, Gandhi A. ENSURE: efficient scheduling and autonomous resource management in serverless environments. In: Proceedings of 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems. 2020, 1−10

[40] Mampage A, Karunasekera S, Buyya R. Deadline-aware dynamic resource management in serverless computing environments. In: Proceedings of the IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing. 2021, 483−492

[41] Qiu J, Zhou Z, Li Y, Li Z, Qian F, Lin H, Gao D, Su H, Miao X, Liu Y, Xu T. vSoC: efficient virtual system-on-chip on heterogeneous hardware. In: Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles. 2024, 558−573

[42] Zhao C, Sun Y, Xiong Y, Krishnamurthy A. Quark: a high-performance secure container runtime for serverless computing. 2023, arXiv preprint arXiv: 2309.12624

[43] Li H, Xu X, Ren J, Dong Y. ACRN: a big little hypervisor for IoT development. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2019, 31−44

[44] Cloud Hypervisor. Run cloud virtual machines securely and efficiently. See www.cloudhypervisor.org/ website, 2024

[45] Dragonball-Sandbox. See github.com/openanolis/dragonball-sandbox website, 2024

[46] Dai Y, Shi Y, Qi Y, Ren J, Wang P. Design and verification of a lightweight reliable virtual machine monitor for a many-core architecture. Frontiers of Computer Science, 2013, 7( 1): 34–43

[47] Overview of DLPAR. See www.ibm.com/docs/sk/powerha-aix/7.2?topic=cluster-overview-dlpar-cod website, 2024

[48] Xen Project. See xenproject.org/ website, 2024

[49] Oracle VM VirtualBox. See www.virtualbox.org/ website, 2024

[50] Hyper-V. See en.wikipedia.org/wiki/Hyper-V website, 2024

[51] VMware. See www.vmware.com/ website, 2024

[52] Du D, Yu T, Xia Y, Zang B, Yan G, Qin C, Wu Q, Chen H. Catalyzer: sub-millisecond startup for serverless computing with initialization-less booting. In: Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems. 2020, 467−481

[53] Liu J, Wang S, Zhou A, Xu J, Yang F. SLA-driven container consolidation with usage prediction for green cloud computing. Frontiers of Computer Science, 2020, 14( 1): 42–52

RIGHTS & PERMISSIONS

Higher Education Press