Performance evaluation of file operations using Mutagen

Abstract

Docker is a vital tool in modern development, enabling the creation, deployment, and execution of applications using containers, thereby ensuring consistency across various environments. However, developers often face challenges, particularly with filesystem complexities and performance bottlenecks when working directly within Docker containers. This is where Mutagen comes into play, significantly enhancing the Docker experience by offering efficient network file synchronization, reducing latency in file operations, and improving overall data transfer rates in containerized environments. By exploring Docker’s architecture, examining Mutagen’s role, and evaluating their combined performance impacts, particularly in terms of file operation speeds and development workflow efficiencies, this research provides a deep understanding of these technologies and their potential to streamline development processes in networked and distributed environments.

Keywords

Mutagen / Docker / OverlayFS / Union mount filesystem / Synchronization / Fio / Rsync

CRediT authorship contribution statement

Mahid Atif Hosain: Writing - original draft, Resources, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Sriram Chellappan: Writing - review & editing, Funding acquisition. Jannatun Noor: Writing - review & editing, Writing - original draft, Visualization, Validation, Supervision, Project administration, Methodology, Investigation, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Docker, Docker, 2024, http://www.docker.com. (Accessed 30 May 2024).

[2] Q. Xu, M. Awasthi, K.T. Malladi, J. Bhimani, J. Yang, M. Annavaram,Docker characterization on high performance SSDs, in: 2017 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS, Santa Rosa, CA, USA, 2017, pp. 133-134, http://dx.doi.org/10.1109/ISPASS.2017.7975282.

[3] S. Nasrin, T.F. Sahryer, A.A. Al Islam, J. Noor,Feature and performance based comparative study on serverless frameworks, in: 2021 24th International Conference on Computer and Information Technology, ICCIT, IEEE, 2021, pp. 1-6.

[4] Z. Kwan, But, it works on my machine, 2018, URL: https://medium.com/hackernoon/but-it-works-on-my-machine-74b6875ab4e7. (Accessed 15 February 2024).

[5] Mutagen, Mutagen, 2024, https://mutagen.io/. (Accessed 15 February 2024).

[6] M. Faiz, U. Shanker,Data synchronization in distributed client-server applications, in: 2016 IEEE International Conference on Engineering and Technology, ICETECH, 2016, pp. 611-616, http://dx.doi.org/10.1109/ICETECH.2016.7569323.

[7] R. Dua, V. Kohli, S. Patil, S. Patil,Performance analysis of union and CoW file systems with docker, in: 2016 International Conference on Computing, Analytics and Security Trends, CAST, Pune, India, 2016, pp. 550-555, http://dx.doi.org/10.1109/CAST.2016.7915029.

[8] N. Brown, Overlay filesystem: whiteouts and opaque directories, 2024, https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt. (Accessed 30 May 2024).

[9] N. Mizusawa, J. Kon, Y. Seki, J. Tao, S. Yamaguchi,Performance improvement of file operations on overlayfs for containers, in: 2018 IEEE International Conference on Smart Computing, SMARTCOMP, Taormina, Italy, 2018, pp. 297-302, http://dx.doi.org/10.1109/SMARTCOMP.2018.00019.

[10] N. Mizusawa, K. Nakazima, S. Yamaguchi,Performance evaluation of file operations on overlayfs, in: 2017 Fifth International Symposium on Computing and Networking, CANDAR, Aomori, Japan, 2017, pp. 597-599, http://dx.doi.org/10.1109/CANDAR.2017.62.

[11] J.N. Mukta, Novel Cloud Storage Ecosystem for Efficient and Secured Multimedia Services, Department of Computer Science and Engineering (CSE), 2023.

[12] S.M. Jain, Cgroups, in: Linux Containers and Virtualization: A Kernel Perspective, A Press, Berkeley, CA, 2020, pp. 45-80, http://dx.doi.org/10.1007/978-1-4842-6283-2_4.

[13] N. Mizusawa, Y. Seki, J. Tao, S. Yamaguchi,A study on I/O performance in highly consolidated container-based virtualized environment on overlayfs with optimized synchronization, in: 2020 14th International Conference on Ubiquitous Information Management and Communication, IMCOM, Taichung, Taiwan, 2020, pp. 1-4, http://dx.doi.org/10.1109/IMCOM48794.2020.9001733.

[14] fio fio, 2024, https://github.com/axboe/fio. (Accessed 15 February 2024).

[15] A. Tridgell, P. Mackerras, et al., Rsync Manual, GNU Public License, Linux Documentation Project, 2008, Man page, URL: https://linux.die.net/man/1/rsync. (Accessed 15 February 2024).

[16] mutagen compose mutagen compose, 2024, https://github.com/mutagen-io/mutagen-compose. (Accessed 15 February 2024).

[17] webinstall webinstall, 2024, https://github.com/webinstall/webi-installers/tree/main. (Accessed 15 February 2024).

[18] Canonical Ltd., Ubuntu - Official docker image, 2023, URL: Accessed 15 December 2023).

[19] P. Acuña, Amazon EC2 container service, in: Deploying Rails with Docker, Kubernetes and ECS, A Press, Berkeley, CA, 2016, pp. 69-98, http://dx.doi.org/10.1007/978-1-4842-2415-1_4.

[20] GitHub, Inc., About large files on GitHub, 2024, URL: https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github. (Accessed 24 September 2024).

[21] J. Noor, N.A. Al-Nabhan, A.A. Al Islam, RemOrphan: Object storage sustainability through removing offline-processed orphan garbage data, IEEE Access (2023).

[22] J. Noor, S.I. Salim, A.A. Al Islam, Strategizing secured image storing and efficient image retrieval through a new cloud framework, J. Netw. Comput. Appl. 192 (2021) 103167.

[23] J. Noor, H.I. Akbar, R.A. Sujon, A.A. Al Islam,Secure processing-aware media storage (SPMS), in: 2017 IEEE 36th International Performance Computing and Communications Conference, IPCCC, IEEE, 2017, pp. 1-8.

[24] F.F. Khan, N.M. Hossain, M.N.H. Shanto, S.B. Anwar, J. Noor, Mitigating ddos attacks using a resource sharing network, in:Proceedings of the 9th International Conference on Networking, Systems and Security, 2022, pp. 1-11.

[25] J. Noor, M.N.H. Shanto, J.J. Mondal, M.G. Hossain, S. Chellappan, A.A. Al Islam, Orchestrating image retrieval and storage over a cloud system, IEEE Trans. Cloud Comput. 11 (2) (2022) 1794-1806.

[26] J. Noor, M.S.I. Sakif, J.J. Mondal, M.R.A. Uday, R.H. Ratul, S. Chellappan, A.A. Al Islam, Sherlock in OSS: A novel approach of content-based searching in object storage system, IEEE Access (2024).

[27] J. Noor, R.H. Ratul, M.S. Basher, J.A. Soumik, S. Sadman, N.J. Rozario, R. Reaz, S. Chellappan, A.A. Al Islam, Secure processing-aware media storage and archival (spmsa), Future Gener. Comput. Syst. 159 (2024) 290-306.

[28] S.D. Dhamdhere, M. Sivakkumar, V. Subramanian, Cloud data security with deep maxout assisted data sanitization and restoration process, High-Confid. Comput. (2024) 100238.

[29] J. Noor, M.G. Hossain, M.A. Alam, A. Uddin, S. Chellappan, A.A. Al Islam,Svload: An automated test-driven architecture for load testing in cloud systems, in: 2018 IEEE Global Communications Conference, GLOBECOM, IEEE, 2018, pp. 1-7.

[30] J. Noor, A.A.Al Islam, ibuck: Reliable and secured image processing middleware for openstack swift, in: 2017 International Conference on Networking, Systems and Security, NSysS, IEEE, 2017, pp. 144-149.

[31] M. Sethunath, Y. Peng, A joint function warm-up and request routing scheme for performing confident serverless computing, High-Confid. Comput. 2 (3) (2022) 100071.