Data cube-based storage optimization for resource-constrained edge computing

Abstract

In the evolving landscape of the digital era, edge computing emerges as an essential paradigm, especially critical for low-latency, real-time applications and Internet of Things (IoT) environments. Despite its advantages, edge computing faces severe limitations in storage capabilities and is fraught with reliability issues due to its resource-constrained nature and exposure to challenging conditions. To address these challenges, this work presents a tailored storage mechanism for edge computing, focusing on space efficiency and data reliability. Our method comprises three key steps: relation factorization, column clustering, and erasure encoding with compression. We successfully reduce the required storage space by deconstructing complex database tables and optimizing data organization within these sub-tables. We further add a layer of reliability through erasure encoding. Comprehensive experiments on TPC-H datasets substantiate our approach, demonstrating storage savings of up to 38.35% and time efficiency improvements by 3.96x in certain cases. Furthermore, our clustering technique shows a potential for additional storage reduction up to 40.41%.

Keywords

Edge computing / Data storage / Reliability / Compression efficiency

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by the State Grid technology project, China (5108-202218280A-2-394-XG), titled "Research and Application on Distributed New Energy Integration and Edge IoT Technology".

References

[1] L. Lin, X. Liao, H. Jin, P. Li, Computation offloading toward edge computing, Proc. IEEE 107 (8) (2019) 1584-1607.

[2] J. Li, G. Luo, N. Cheng, Q. Yuan, Z. Wu, S. Gao, Z. Liu, An end-to-end load balancer based on deep learning for vehicular network traffic control, IEEE Internet Things J. 6 (1) (2018) 953-966.

[3] S. Wang, Y. Zhao, J. Xu, J. Yuan, C.-H. Hsu, Edge server placement in mobile edge computing, J. Parallel Distrib. Comput. 127 (2019) 160-168.

[4] TPC-H benchmark, 2018, http://www.tpc.org/tpch. Accessed: 2023-08-29.

[5] W.Z. Khan, E. Ahmed, S. Hakak, I. Yaqoob, A. Ahmed, Edge computing: a survey, Future Gener. Comput. Syst. 97 (2019) 219-235, http://dx.doi.org/10.1016/j.future.2019.02.050.

[6] C. Esposito, A. Castiglione, F. Pop, K.-K.R. Choo, Challenges of connecting edge and cloud computing: a security and forensic perspective, IEEE Cloud Comput. 4 (2) (2017) 13-17.

[7] W. Shi, S. Dustdar, The promise of edge computing, Computer 49 (5) (2016) 78-81.

[8] B. Jia, H. Hu, Y. Zeng, T. Xu, Y. Yang, Double-matching resource allocation strategy in fog computing networks based on cost efficiency, J. Commun. Netw. 20 (3) (2018) 237-246.

[9] L. Gu, D. Zeng, S. Guo, A. Barnawi, Y. Xiang, Cost efficient resource management in fog computing supported medical cyber-physical system, IEEE Trans. Emerg. Top. Comput. 5 (1) (2015) 108-119.

[10] M.A. Hassan, M. Xiao, Q. Wei, S. Chen,Help your mobile applications with fog computing, in: 2015 12th Annual IEEE International Conference on Sensing, Communication, and Networking-Workshops, SECON Workshops, IEEE, 2015, pp. 1-6.

[11] K. Hong, D. Lillethun, U. Ramachandran, B. Ottenwälder, B. Koldehofe, Mobile fog: A programming model for large-scale applications on the internet of things,in:Proceedings of the Second ACM SIGCOMM Workshop on Mobile Cloud Computing, 2013, pp. 15-20.

[12] F. Bonomi, R. Milito, P. Natarajan, J. Zhu, Fog computing: a platform for internet of things and analytics,in:Big Data and Internet of Things: a Roadmap for Smart Environments, Springer, 2014, pp. 169-186.

[13] M. Aazam, E.-N. Huh,Fog computing micro datacenter based dynamic resource estimation and pricing model for IoT, in: 2015 IEEE 29th International Conference on Advanced Information Networking and Applications, IEEE, 2015, pp. 687-694.

[14] C.T. Do, N.H. Tran, C. Pham, M.G.R. Alam, J.H. Son, C.S. Hong,A proximal algorithm for joint resource allocation and minimizing carbon footprint in geo-distributed fog computing, in: 2015 International Conference on Information Networking, ICOIN, IEEE, 2015, pp. 324-329.

[15] X. Lin, G. Lu, F. Douglis, P. Shilane, G. Wallace, Migratory compression: Coarse-grained data reordering to improve compressibility,in:12th USENIX Conference on File and Storage Technologies, FAST 14, 2014, pp. 256-273.

[16] P. Mach, Z. Becvar, Mobile edge computing: a survey on architecture and computation offloading, IEEE Commun. Surv. Tutorials 19 (3) (2017) 1628-1656.

[17] Y. Mao, C. You, J. Zhang, K. Huang, K.B. Letaief, A survey on mobile edge computing: the communication perspective, IEEE Commun. Surv. Tutorials 19 (4) (2017) 2322-2358.

[18] S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, W. Wang, A survey on mobile edge networks: Convergence of computing, caching and communications, IEEE Access 5 (2017) 6757-6779.

[19] N. Abbas, Y. Zhang, A. Taherkordi, T. Skeie, Mobile edge computing: a survey, IEEE Internet Things J. 5 (1) (2017) 450-465.

[20] K. Peng, V. Leung, X. Xu, L. Zheng, J. Wang, Q. Huang, et al., A survey on mobile edge computing: Focusing on service adoption and provision, Wirel. Commun. Mob. Comput. 2018 (2018).

[21] K. Toczé, S. Nadjm-Tehrani, A taxonomy for management and optimization of multiple resources in edge computing, Wirel. Commun. Mob. Comput. 2018 (2018).

[22] T.L. Duc, R.G. Leiva, P. Casari, P.-O. Östberg, Machine learning methods for reliable resource provisioning in edge-cloud computing: a survey, ACM Comput. Surv. 52 (5) (2019) 1-39.

[23] C.-H. Hong, B. Varghese, Resource management in fog/edge computing: a survey on architectures, infrastructure, and algorithms, ACM Comput. Surv. 52 (5) (2019) 1-37.

[24] M. Ghobaei-Arani, A. Souri, A.A. Rahmanian, Resource management approaches in fog computing: a comprehensive review, J. Grid Comput. 18 (1) (2020) 1-42.

[25] J. Santos, T. Wauters, B. Volckaert, F. De Turck, Resource provisioning in fog computing: From theory to practice, Sensors 19 (10) (2019) 2238.

[26] J. Ren, D. Zhang, S. He, Y. Zhang, T. Li, A survey on end-edge-cloud or- chestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet, ACM Comput. Surv. 52 (6) (2019) 1-36.

[27] B. Varghese, N. Wang, D. Bermbach, C.-H. Hong, E.D. Lara, W. Shi, C. Stewart, A survey on edge performance benchmarking, ACM Comput. Surv. 54 (3) (2021) 1-33.

[28] F. Chen, P. Li, C. Wu, S. Guo, Hare: Exploiting inter-job and intra-job parallelism of distributed machine learning on heterogeneous GPUs,in:Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 253-264, http://dx.doi.org/10.1145/3502181.3531462.

[29] F. Chen, P. Li, D. Zeng, S. Guo, Edge-assisted short video sharing with guaranteed quality-of-experience, IEEE Trans. Cloud Comput. 11 (1) (2023) 13-24, http://dx.doi.org/10.1109/TCC.2021.3067834.

[30] X. Pang, Z. Wang, D. Liu, J.C.S. Lui, Q. Wang, J. Ren, Towards personalized privacy-preserving truth discovery over crowdsourced data streams, IEEE/ACM Trans. Netw. 30 (1) (2022) 327-340, http://dx.doi.org/10.1109/TNET.2021.3110052.

[31] Z. Wang, K. Liu, J. Hu, J. Ren, H. Guo, W. Yuan, AttrLeaks on the edge: Exploiting information leakage from privacy-preserving co-inference, Chin. J. Electron. 32 (1) (2023) 1-12, http://dx.doi.org/10.23919/cje.2022.00.031.

[32] Z. Shen, J. Wu, X. Jiang, Y. Zhang, L. Ju, Z. Jia, PRAP-PIM: A weight pattern reusing aware pruning method for reram-based PIM dnn accelerators, High-Confid. Comput. 3 (2) (2023) 100123, http://dx.doi.org/10.1016/j.hcc.2023.100123.

[33] M. Xu, Y. Guo, Q. Hu, Z. Xiong, D. Yu, X. Cheng, A trustless architecture of blockchain-enabled metaverse, High-Confid. Comput. 3 (1) (2023) 100088, http://dx.doi.org/10.1016/j.hcc.2022.100088.

[34] M. Cao, Y. Zhang, Z. Ma, M. Zhao,C2S: Class-aware client selection for effective aggregation in federated learning, High-Confid. Comput. 2 (3) (2022) 100068, http://dx.doi.org/10.1016/j.hcc.2022.100068.

[35] Y. Guo, Z. Wan, X. Cheng, When blockchain meets smart grids: A comprehensive survey, High-Confid. Comput. 2 (2) (2022) 100059, http://dx.doi.org/10.1016/j.hcc.2022.100059.

[36] K. Peguero, X. Cheng, CSRF protection in JavaScript frameworks and the security of JavaScript applications, High-Confid. Comput. 1 (2) (2021) 100035, http://dx.doi.org/10.1016/j.hcc.2021.100035.

[37] K. Peguero, X. Cheng, Electrolint and security of electron applications, High-Confid. Comput. 1 (2) (2021) 100032, http://dx.doi.org/10.1016/j.hcc.2021.100032.

[38] Q. Xia, W. Ye, Z. Tao, J. Wu, Q. Li, A survey of federated learning for edge computing: Research problems and solutions, High-Confid. Comput. 1 (1) (2021) 100008, http://dx.doi.org/10.1016/j.hcc.2021.100008.

[39] Y. He, C. Zhang, B. Wu, Z. Geng, K. Xiao, H. Li, A trusted architecture for EV shared charging based on blockchain technology, High-Confid. Comput. 1 (2) (2021) 100001, http://dx.doi.org/10.1016/j.hcc.2021.100001.

[40] Z. Wang, Q. Hu, Y. Wang, Y. Xiao, Transaction pricing mechanism design and assessment for blockchain, High-Confid. Comput. 2 (1) (2022) 100044, http://dx.doi.org/10.1016/j.hcc.2021.100044.

[41] Y. Chen, H. Chen, Y. Zhang, M. Han, M. Siddula, Z. Cai, A survey on blockchain systems: Attacks, defenses, and privacy preservation, High-Confid. Comput. 2 (2) (2022) 100048, http://dx.doi.org/10.1016/j.hcc.2021.100048.

[42] D. Zhao, Z. Li, H. Ding, Z. Zhang, Z. Li, Research and design of CRT-based homomorphic ciphertext database system, High-Confid. Comput. 2 (4) (2022) 100074, http://dx.doi.org/10.1016/j.hcc.2022.100074.

[43] J. Zou, D. He, S. Bi, L. Wu, Z. Liu, C. Peng,A certificateless multi-receiver encryption scheme based on SM2 signature algorithm, High-Confid. Comput. 3 (1) (2023) 100103, http://dx.doi.org/10.1016/j.hcc.2023.100103.

[44] J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, et al., Oceanstore: an architecture for global-scale persistent storage, Oper. Syst. Rev. 34 (5) (2000) 190-201.

[45] N. Joukov, A.M. Krishnakumar, C. Patti, A. Rai, S. Satnur, A. Traeger, E. Zadok,RAIF: Redundant array of independent filesystems, in: 24th IEEE Conference on Mass Storage Systems and Technologies, MSST 2007, IEEE, 2007, pp. 199-214.

[46] V. Bohossian, C.C. Fan, P.S. LeMahieu, M.D. Riedel, L. Xu, J. Bruck, Computing in the rain: a reliable array of independent nodes, IEEE Trans. Parallel Distrib. Syst. 12 (2) (2001) 99-114.

[47] P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, S. Sankar, Row-diagonal parity for double disk failure correction, in:Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 2004, pp. 1-14.

[48] L. Xu, V. Bohossian, J. Bruck, D.G. Wagner, Low-density mds codes and factors of complete graphs, IEEE Trans. Inf. Theory 45 (6) (1999) 1817-1826.

[49] M. Luby, D. Zuckermank, An Xor-Based Erasure-Resilient Coding Scheme, Tech Report, Tech. Rep., Citeseer, 1995.

[50] I.S. Reed, G. Solomon, Polynomial codes over certain finite fields, J. Soc. Ind. Appl. Math. 8 (2) (1960) 300-304.

[51] M. Blaum, R.M. Roth, On lowest density mds codes, IEEE Trans. Inform. Theory 45 (1) (1999) 46-59.

[52] L. Xu, J. Bruck, X-Code: Mds array codes with optimal encoding, IEEE Trans. Inform. Theory 45 (1) (1999) 272-276.

[53] J.S. Plank, L. Xu, Optimizing Cauchy reed-solomon codes for faulttolerant network storage applications, in: Fifth IEEE International Symposium on Network Computing and Applications, NCA’06, IEEE, 2006, pp. 173-180.