Collaborative multi-granularity distributed registry planning for fast container image pulling

Abstract

The increasing popularity of container technology raises significant challenges in efficiently storing millions of container images in registries to enable fast on-demand image pulling. This is further complicated by (1) registries are geographically distributed, with independent and heterogeneous storage resources; (2) container images are pulled in layers, but can be stored at different levels of granularity, i.e., layer-level or file-level, each with varying storage requirement and pulling latency. To address the above challenges, we propose MIS, a multi-granularity image storage strategy, for distributed registries to determine the storage granularity and schedule image storage collaboratively, aiming to reduce the image pulling latency while improving the storage utilization. We formulate the image storage problem into a nonlinear mixed-integer programming form with NP-hardness by incorporating both layer-level and file-level storage constraints. We propose a low computational complexity algorithm via randomized rounding with a guaranteed approximation ratio. Extensive experimental results demonstrate the effectiveness of our strategy, with image pulling latency reductions of 28.67%, 21.69%, and 28.94% respectively compared to the state-of-the-art solutions.

Graphical abstract

Keywords

edge computing / container image pulling / microservice deployment

References

[1] Liu X, Zhao Y, Liu S, Li X, Zhu Y, Liu X, Jin X . MuxFlow: efficient GPU sharing in production-level clusters with more than 10000 GPUs. Science China Information Sciences, 2024, 67( 12): 222101

[2] Shan C, Gao R, Yang Z, Zhang W, Xia Y . ControlService: a containerized solution for control-algorithm-as-a-service in cloud control systems. Science China Information Sciences, 2024, 67( 8): 182201

[3] Kang H, Le M, Tao S. Container and microservice driven design for cloud infrastructure DevOps. In: Proceedings of the IEEE International Conference on Cloud Engineering. 2016, 202−211

[4] Zeng R, Hou X, Zhang L, Li C, Zheng W, Guo M . Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities. Frontiers of Computer Science, 2022, 16( 6): 166106

[5] Felter W, Ferreira A, Rajamony R, Rubio J. An updated performance comparison of virtual machines and Linux containers. In: Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software. 2015, 171−172

[6] Anwar A, Rupprecht L, Skourtis D, Tarasov V . Challenges in storing docker images. Login - The Usenix Magazine, 2019, 44( 3): 32–36

[7] Li S, Zhou A, Ma X, Xu M, Wang S. Commutativity-guaranteed docker image reconstruction towards effective layer sharing. In: Proceedings of the ACM Web Conference. 2022, 3358−3366

[8] Zhao N, Tarasov V, Albahar H, Anwar A, Rupprecht L, Skourtis D, Paul A K, Chen K, Butt A R . Large-scale analysis of docker images and performance implications for container storage systems. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 4): 918–930

[9] Littley M, Anwar A, Fayyaz H, Fayyaz Z, Tarasov V, Rupprecht L, Skourtis D, Mohamed M, Ludwig H, Cheng Y, Butt A R. Bolt: towards a scalable docker registry via hyperconvergence. In: Proceedings of the 12th IEEE International Conference on Cloud Computing. 2019, 358−366

[10] Zhao N, Albahar H, Abraham S, Chen K, Tarasov V, Skourtis D, Rupprecht L, Anwar A, Butt A R. DupHunter: flexible high-performance deduplication for docker registries. In: Proceedings of the USENIX Annual Technical Conference. 2020, 53

[11] Anwar A, Mohamed M, Tarasov V, Littley M, Rupprecht L, Cheng Y, Zhao N, Skourtis D, Warke A S, Ludwig H, Hildebrand D, Butt A R. Improving docker registry design based on production workload analysis. In: Proceedings of the 16th USENIX Conference on File and Storage Technologies. 2018, 265−278

[12] Xu X, Wu F, Bilal M, Xia X, Dou W, Yao L, Zhong W . XRL-SHAP-Cache: an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks. Science China Information Sciences, 2024, 67( 7): 170303

[13] Harter T, Salmon B, Liu R, Arpaci-Dusseau A C, Arpaci-Dusseau R H. Slacker: fast distribution with lazy docker containers. In: Proceedings of the 14th USENIX Conference on File and Storage Technologies. 2016, 181−195

[14] Fu S, Mittal R, Zhang L, Ratnasamy S. Fast and efficient container startup at the edge via dependency scheduling. In: Proceedings of the 3rd USENIX Workshop on Hot Topics in Edge Computing. 2020

[15] Hua Z, Yu Y, Gu J, Xia Y, Chen H, Zang B . TZ-container: protecting container from untrusted OS with ARM TrustZone. Science China Information Sciences, 2021, 64( 9): 192101

[16] Merkel D . Docker: lightweight Linux containers for consistent development and deployment. Linux Journal, 2014, 2014( 239): 2

[17] Tang Z, Lou J, Jia W . Layer dependency-aware learning scheduling algorithms for containers in mobile edge computing. IEEE Transactions on Mobile Computing, 2023, 22( 6): 3444–3459

[18] Gu L, Zeng D, Hu J, Jin H, Guo S, Zomaya A Y. Exploring layered container structure for cost efficient microservice deployment. In: Proceedings of the IEEE Conference on Computer Communications. 2021, 1−9

[19] Nathan S, Ghosh R, Mukherjee T, Narayanan K. CoMICon: a co-operative management system for docker container images. In: Proceedings of the IEEE International Conference on Cloud Engineering. 2017, 116−126

[20] Zhao N, Tarasov V, Albahar H, Anwar A, Rupprecht L, Skourtis D, Warke A S, Mohamed M, Butt A R. Large-scale analysis of the docker hub dataset. In: Proceedings of the IEEE International Conference on Cluster Computing. 2019, 1−10

[21] Zhao N, Tarasov V, Anwar A, Rupprecht L, Skourtis D, Warke A, Mohamed M, Butt A. Slimmer: weight loss secrets for docker registries. In: Proceedings of the 12th IEEE International Conference on Cloud Computing. 2019, 517−519

[22] Smet P, Dhoedt B, Simoens P . Docker layer placement for on-demand provisioning of services on edge clouds. IEEE Transactions on Network and Service Management, 2018, 15( 3): 1161–1174

[23] Bauer S, Wiedner F, Jaeger B, Emmerich P, Carle G. Scalable TCP throughput limitation monitoring. In: Proceedings of the IFIP/IEEE International Symposium on Integrated Network Management. 2021, 410−416

[24] Young N E. Greedy set-cover algorithms: 1974−1979; Chvatal, Johnson, Lovász, Stein. In: Kao M Y, ed. Encyclopedia of Algorithms. New York: Springer, 2008, 379−381

[25] Mitzenmacher M, Upfal E. Probability and Computing: Randomized Algorithms and Probabilistic Analysis. New York: Cambridge University Press, 2005

[26] Chen J L, Liaqat D, Gabel M, de Lara E. Starlight: fast container provisioning on the edge and over the WAN. In: Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation. 2022, 35−50

[27] Wang R, Mou X, Wo T, Zhang M, Liu Y, Wang T, Liu P, Yan J, Liu X . ACbot: an IIoT platform for industrial robots. Frontiers of Computer Science, 2025, 19( 4): 194203

[28] Wang K, Wu S, Cui Y, Huang Z, Fan H, Jin H . System log isolation for containers. Frontiers of Computer Science, 2025, 19( 5): 195106

[29] Morabito R, Kjällman J, Komu M. Hypervisors vs. lightweight virtualization: a performance comparison. In: Proceedings of the IEEE International Conference on Cloud Engineering. 2015, 386−393

[30] Thalheim J, Bhatotia P, Fonseca P, Kasikci B. CNTR: lightweight OS containers. In: Proceedings of the USENIX Annual Technical Conference. 2018, 199−212

[31] Park M, Bhardwaj K, Gavrilovska A. Toward lighter containers for the edge. In: Proceedings of the 3rd USENIX Workshop on Hot Topics in Edge Computing. 2020

[32] Lou J, Luo H, Tang Z, Jia W, Zhao W . Efficient container assignment and layer sequencing in edge computing. IEEE Transactions on Services Computing, 2023, 16( 2): 1118–1131

[33] Gu L, Chen Z, Xu H, Zeng D, Li B, Jin H. Layer-aware collaborative microservice deployment toward maximal edge throughput. In: Proceedings of the IEEE Conference on Computer Communications. 2022, 71−79

[34] Wang S, Ding Z, Jiang C . Elastic scheduling for microservice applications in clouds. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 1): 98–115

[35] Li Y, An B, Ma J, Cao D. Comparison between chunk-based and layer-based container image storage approaches: an empirical study. In: Proceedings of the IEEE International Conference on Service-Oriented System Engineering. 2019, 197−202

[36] Li H, Yuan Y, Du R, Ma K, Liu L, Hsu W. DADI: block-level image service for agile and elastic application deployment. In: Proceedings of the USENIX Annual Technical Conference. 2020, 50

[37] Shanmugam K, Golrezaei N, Dimakis A G, Molisch A F, Caire G . FemtoCaching: wireless content delivery through distributed caching helpers. IEEE Transactions on Information Theory, 2013, 59( 12): 8402–8413

[38] Golrezaei N, Shanmugam K, Dimakis A G, Molisch A F, Caire G. FemtoCaching: wireless video content delivery through distributed caching helpers. In: Proceedings of the IEEE INFOCOM. 2012, 1107−1115

[39] Poularakis K, Iosifidis G, Argyriou A, Koutsopoulos I, Tassiulas L. Caching and operator cooperation policies for layered video content delivery. In: Proceedings of the 35th Annual IEEE Conference on Computer Communications. 2016, 1−9

[40] Applegate D, Archer A, Gopalakrishnan V, Lee S, Ramakrishnan K K . Optimal content placement for a large-scale VoD system. IEEE/ACM Transactions on Networking, 2016, 24( 4): 2114–2127

[41] Liu Y, Mao Y, Shang X, Liu Z, Yang Y. Distributed cooperative caching in unreliable edge environments. In: Proceedings of the IEEE Conference on Computer Communications. 2022, 1049−1058

[42] Skourtis D, Rupprecht L, Tarasov V, Megiddo N. Carving perfect layers out of Docker images. In: Proceedings of the 11th USENIX Workshop on Hot Topics in Cloud Computing. 2019

RIGHTS & PERMISSIONS

The Author(s) 2025. This article is published with open access at link.springer.com and journal.hep.com.cn