SoK: Decentralized storage network

Abstract

Decentralized Storage Networks (DSNs) represent a paradigm shift in data storage methodology, distributing and housing data across multiple network nodes rather than relying on a centralized server or data center architecture. The fundamental objective of DSNs is to enhance security, reinforce reliability, and mitigate censorship risks by eliminating a single point of failure. Leveraging blockchain technology for functions such as access control, ownership validation, and transaction facilitation, DSN initiatives aim to provide users with a robust and secure alternative to traditional centralized storage solutions. This paper conducts a comprehensive analysis of the developmental trajectory of DSNs, focusing on key components such as Proof of Storage protocols, consensus algorithms, and incentive mechanisms. Additionally, the study explores recent optimization tactics, encountered challenges, and potential avenues for future research, thereby offering insights into the ongoing evolution and advancement within the DSN domain.

Keywords

Decentralized Storage Network / Blockchain / Proof of Storage / Consensus algorithm

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This study was partially supported by the National Key R&D Program of China (2022YFB4501000), the National Natural Science Foundation of China (62232010, 62302266, and U23A20302), Shandong Science Fund for Excellent Young Scholars (2023HWYQ-008), and Shandong Science Fund for Key Fundamental Research Project (ZR2022ZD02).

References

[1] M.R. Palankar, A. Iamnitchi, M. Ripeanu, S. Garfinkel, Amazon S3 for science grids: a viable solution? in: Proceedings of the 2008 International Workshop on Data-Aware Distributed Computing, 2008, pp. 55-64.

[2] S. Challita, F. Zalila, C. Gourdin, P. Merle, A precise model for google cloud platform, in: 2018 IEEE International Conference on Cloud Engineering, IC2E, IEEE, 2018, pp. 177-183.

[3] G. Zhang, M. Ravishankar, Exploring vendor capabilities in the cloud environment: A case study of alibaba cloud computing, Inf. Manage. 56 (3) (2019) 343-355.

[4] X. Cheng, M. Xu, R. Pan, D. Yu, C. Wang, X. Xiao, W. Lyu, Meta computing, IEEE Netw. (2023).

[5] J. Ernstberger, J. Lauinger, F. Elsheimy, L. Zhou, S. Steinhorst, R. Canetti, A. Miller, A. Gervais, D. Song, SoK: Data sovereignty, 2023, Cryptology ePrint Archive.

[6] D. Vorick, L. Champine, Sia: Simple decentralized storage, Retrieved May, vol. 8, 2014, p. 2018.

[7] I. Storj Labs, Storj: A decentralized cloud storage network framework, 2018.

[8] S. Team, SWARM-storage and communication infrastructure for a self-sovereign digital society, 2021.

[9] Protocol Labs, Filecoin: A decentralized storage network, 2017, Retrieved from: https://filecoin.io/filecoin.pdf.

[10] C. Reed, Information in the cloud: ownership, control and accountability, in: Privacy and Legal Issues in Cloud Computing, Edward Elgar Publishing, 2015, pp. 139-159.

[11] AWS, AWS docs, 2023, https://aws.amazon.com/what-is/cloud-storage/.

[12] I. Clarke, O. Sandberg, B. Wiley, T.W. Hong, Freenet: A distributed anonymous information storage and retrieval system,in:Designing Privacy Enhancing Technologies: International Workshop on Design Issues in Anonymity and Unobservability Berkeley, CA, USA, July 25-26, 2000 Proceedings, Springer, 2001, pp. 46-66.

[13] A. Muthitacharoen, R. Morris, T.M. Gil, B. Chen, Ivy: A read/write peer-to-peer file system, Oper. Syst. Rev. 36 (SI) (2002) 31-44.

[14] F. Giroire, J. Monteiro, S. Pérennes, P2p storage systems: How much locality can they tolerate? in: 2009 IEEE 34th Conference on Local Computer Networks, IEEE, 2009, pp. 320-323.

[15] C. Williams, P. Huibonhoa, J. Holliday, A. Hospodor, T. Schwarz, Redundancy management for P2P storage, in: Seventh IEEE International Symposium on Cluster Computing and the Grid, CCGrid’07, IEEE, 2007, pp. 15-22.

[16] I. Osipkov, P. Wang, N. Hopper, Robust accounting in decentralized P2P storage systems, in: 26th IEEE International Conference on Distributed Computing Systems, ICDCS’06, IEEE, 2006, p. 14.

[17] D. Trautwein, A. Raman, G. Tyson, I. Castro, W. Scott, M. Schubotz, B. Gipp, Y. Psaras, Design and evaluation of IPFS: a storage layer for the decentralized web,in:Proceedings of the ACM SIGCOMM 2022 Conference, 2022, pp. 739-752.

[18] P. Maymounkov, D. Mazieres, Kademlia: A peer-to-peer information system based on the xor metric,in: International Workshop on Peer-To-Peer Systems, Springer, 2002, pp. 53-65.

[19] J. Benet, Ipfs-content addressed, versioned, p2p file system, 2014, arXiv preprint arXiv:1407.3561.

[20] E. Daniel, F. Tschorsch, IPFS and friends: A qualitative comparison of next generation peer-to-peer data networks, IEEE Commun. Surv. Tutor. 24 (1) (2022) 31-52.

[21] M. Song, J. Han, H. Eom, Y. Son, IPFSz: An efficient data compression scheme in InterPlanetary File System, IEEE Access 10 (2022) 122601-122611.

[22] J. Sun, X. Yao, S. Wang, Y. Wu, Blockchain-based secure storage and access scheme for electronic medical records in IPFS, IEEE Access 8 (2020) 59389-59401.

[23] H. Guo, M. Xu, J. Zhang, C. Liu, D. Yu, S. Dustdar, X. Cheng, FileDAG: A multi-version decentralized storage network built on DAG-based blockchain, IEEE Trans. Comput. (2023).

[24] I. Keidar, E. Kokoris-Kogias, O. Naor, A. Spiegelman, All you need is dag, in:Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, 2021, pp. 165-175.

[25] G. Ateniese, R. Burns, R. Curtmola, J. Herring, L. Kissner, Z. Peterson, D. Song, Provable data possession at untrusted stores, in:Proceedings of the 14th ACM Conference on Computer and Communications Security, 2007, pp. 598-609.

[26] A. Juels, B.S. Kaliski Jr., PORs: Proofs of retrievability for large files,in:Proceedings of the 14th ACM Conference on Computer and Communications Security, 2007, pp. 584-597.

[27] A.G. Dimakis, V. Prabhakaran, K. Ramchandran, Decentralized erasure codes for distributed networked storage, IEEE Trans. Inform. Theory 52 (6) (2006) 2809-2816.

[28] Q. Zheng, S. Xu, Secure and efficient proof of storage with deduplication, in:Proceedings of the Second ACM Conference on Data and Application Security and Privacy, 2012, pp. 1-12.

[29] Storj, Storj docs, 2023, https://docs.storj.io/learn/concepts/satellite.

[30] J. Benet, D. Dalrymple, N. Greco, Proof of replication, Protocol Labs, July, vol. 27, 2017, p. 20.

[31] H. Qi, Y. Cheng, M. Xu, D. Yu, H. Wang, W. Lyu, Split: A hash-based memory optimization method for zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK), IEEE Trans. Comput. (2023).

[32] I. Damgård, C. Ganesh, C. Orlandi, Proofs of replicated storage without timing assumptions, in: Advances in Cryptology-CRYPTO 2019: 39th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2019, Proceedings, Part I 39, Springer, 2019, pp. 355-380.

[33] E. Cecchetti, B. Fisch, I. Miers, A. Juels, PIEs: Public incompressible encodings for decentralized storage,in:Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019, pp. 1351-1367.

[34] H. Yuan, X. Chen, G. Xu, J. Ning, J.K. Liu, R.H. Deng, Efficient and verifiable proof of replication with fast fault localization, in: IEEE INFOCOM 2021-IEEE Conference on Computer Communications, IEEE, 2021, pp. 1-10.

[35] C. Zhang, X. Li, M.H. Au, ePoSt: Practical and client-friendly proof of storage-time, IEEE Trans. Inf. Forensics Secur. 18 (2023) 1052-1063.

[36] T. Moran, I. Orlov, Simple proofs of space-time and rational proofs of storage, in: Advances in Cryptology-CRYPTO 2019: 39th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2019, Proceedings, Part I 39, Springer, 2019, pp. 381-409.

[37] G. Ateniese, L. Chen, M. Etemad, Q. Tang, Proof of storage-time: Efficiently checking continuous data availability, 2020, Cryptology ePrint Archive.

[38] R. Rabaninejad, B. Abdolmaleki, G. Malavolta, A. Michalas, A. Nabizadeh, stoRNA: Stateless transparent proofs of storage-time, 2023, Cryptology ePrint Archive.

[39] R. Rabaninejad, B. Liu, A. Michalas, PoRt: Non-interactive continuous availability proof of replicated storage,in:Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, 2023, pp. 270-279.

[40] M. Xu, Y. Guo, C. Liu, Q. Hu, D. Yu, Z. Xiong, D. Niyato, X. Cheng, Exploring blockchain technology through a modular lens: A survey, 2023, arXiv preprint arXiv:2304.08283.

[41] S. King, S. Nadal, Ppcoin: Peer-to-peer crypto-currency with proof-of-stake, 2012, self-published paper, August, vol. 19, no. 1.

[42] F. Victor, B.K. Lüders, Measuring ethereum-based erc20 token networks, in: Financial Cryptography and Data Security: 23rd International Conference, FC 2019, Frigate Bay, St. Kitts and Nevis, February 18-22, 2019, Revised Selected Papers 23, Springer, 2019, pp. 113-129.

[43] F. Saleh, Blockchain without waste: Proof-of-stake, Rev. Financ. Stud. 34 (3) (2021) 1156-1190.

[44] Filecoin, Filecoin docs, 2023, https://docs.filecoin.io/.

[45] Filecoin, Filecoin spec, 2023, https://spec.filecoin.io/.

[46] V.H. Lakhani, L. Jehl, R. Hendriksen, V. Estrada-Galinanes, Fair incentivization of bandwidth sharing in decentralized storage networks, in: 2022 IEEE 42nd International Conference on Distributed Computing Systems Workshops, ICDCSW, IEEE, 2022, pp. 39-44.

[47] V.H. Lakhani, L. Jehl, G. Ishmaev, V. Estrada-Galiñanes, Tit-for-token: fair rewards for moving data in decentralized storage networks, 2023, arXiv preprint arXiv:2307.02231.

[48] L. Rizzo, Effective erasure codes for reliable computer communication protocols, ACM SIGCOMM Comput. Commun. Rev. 27 (2) (1997) 24-36.

[49] H. Guo, M. Xu, J. Zhang, C. Liu, R. Ranjan, D. Yu, X. Cheng, BFT-DSN: A Byzantine fault tolerant decentralized storage network, IEEE Trans. Comput. (2024) 1-13.

[50] J.R. Douceur, The sybil attack, in: International Workshop on Peer-To-Peer Systems, Springer, 2002, pp. 251-260.

[51] B. Prünster, A. Marsalek, T. Zefferer, Total eclipse of the heart-disrupting the {InterPlanetary} file system, in:31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 3735-3752.

[52] S. Wang, Y. Zhang, Y. Zhang, A blockchain-based framework for data sharing with fine-grained access control in decentralized storage systems, Ieee Access 6 (2018) 38437-38450.

[53] M. Steichen, B. Fiz, R. Norvill, W. Shbair, R. State, Blockchain-based, decentralized access control for IPFS, in: 2018 Ieee International Conference on Internet of Things (IThings) and Ieee Green Computing and Communications (GreenCom) and Ieee Cyber, Physical and Social Computing (CPSCom) and Ieee Smart Data (SmartData), IEEE, 2018, pp. 1499-1506.

[54] G. Ateniese, K. Fu, M. Green, S. Hohenberger, Improved proxy re-encryption schemes with applications to secure distributed storage, ACM Trans. Inf. Syst. Secur. 9 (1) (2006) 1-30.

[55] M. Xu, J. Zhang, H. Guo, X. Cheng, D. Yu, Q. Hu, Y. Li, Y. Wu, FileDES: A secure, scalable and succinct decentralized encrypted storage network, 2024, Cryptology ePrint Archive, Paper 2024/182, [Online]. Available: https://eprint.iacr.org/2024/182.

[56] J. He, D. Zheng, R. Guo, Y. Chen, K. Li, X. Tao, Efficient identity-based proxy re-encryption scheme in blockchain-assisted decentralized storage system, Int. J. Netw. Secur. 23 (5) (2021) 776-790.

[57] J. Kan, J. Zhang, D. Liu, X. Huang, Proxy re-encryption scheme for decentralized storage networks, Appl. Sci. 12 (9) (2022) 4260.

[58] Protocol Labs Nft. storage docs, 2023, https://nft.storage/docs/.

[59] OpenSea, OpenSea, 2023, https://opensea.io/.

[60] M. Grigore, S. Kassab, Filecoin has it: An ecosystem overview, 2022, https://messari.io/report/filecoin-has-it-an-ecosystem-overview.

[61] M. Fenwick, P. Jurcys, The contested meaning of Web3 and why it matters for (IP) lawyers, 2022, Available at SSRN 4017790.

[62] L.-H. Lee, T. Braud, P. Zhou, L. Wang, D. Xu, Z. Lin, A. Kumar, C. Bermejo, P. Hui, All one needs to know about metaverse: A complete survey on technological singularity, virtual ecosystem, and research agenda, 2021, arXiv preprint arXiv:2110.05352.

[63] Y. Cheng, Y. Guo, M. Xu, Q. Hu, D. Yu, X. Cheng, An adaptive and modular blockchain enabled architecture for a decentralized metaverse, IEEE J. Sel. Areas Commun. (2023) 1.

[64] M. Xu, Y. Guo, Q. Hu, Z. Xiong, D. Yu, X. Cheng, A trustless architecture of blockchain-enabled metaverse, High-Confidence Comput. 3 (1) (2023) 100088.

[65] Mona, Mona docs, 2023, https://docs.monaverse.com/.

[66] Volaverse, Volaverse, 2023, https://www.volaverse.com/.

[67] Huddle01, Huddle01, 2023, https://huddle01.com/.

[68] Filecoin, Filecoin for media, video, gaming, and more, 2021, https://filecoin.io/blog/posts/filecoin-for-media-video-gaming-and-more/.

[69] D. Petkanics, E. Tang, Livepeer whitepaper, 2023, https://github.com/livepeer/wiki/blob/master/WHITEPAPER.md.

[70] Voodfy, Voodfy, 2021, https://github.com/Voodfy.

[71] A.A. Khan, A.A. Wagan, A.A. Laghari, A.R. Gilal, I.A. Aziz, B.A. Talpur, BIoMT: A state-of-the-art consortium serverless network architecture for healthcare system using blockchain smart contracts, IEEE Access 10 (2022) 78887-78898.

[72] G. Adityaa, V. Lavanya, A decentralized storage system for 3D medical data with dynamic AES and AES-GCM encryption, in:Recent Developments in Electronics and Communication Systems: Proceedings of the First International Conference on Recent Developments in Electronics and Communication Systems, Vol. 32, RDECS-2022, IOS Press, 2023, p. 269.

[73] H. Subramanian, S. Subramanian, Improving diagnosis through digital pathology: Proof-of-concept implementation using smart contracts and decentralized file storage, J. Med. Internet Res. 24 (3) (2022) e34207.

[74] B. Supernova, IPEHR, 2023, https://github.com/bsn-si/IPEHR-gateway.

[75] M. Smith, A. Castro, M. Rahouti, M. Ayyash, L. Santana, ScreenCoin: A blockchain-enabled decentralized Ad network, in: 2022 IEEE International Conference on Omni-Layer Intelligent Systems, COINS, IEEE, 2022, pp. 1-6.

[76] J. Alwen, J. Blocki, K. Pietrzak, Depth-robust graphs and their cumulative memory complexity, in: Annual International Conference on the Theory and Applications of Cryptographic Techniques, Springer, 2017, pp. 3-32.

[77] B. Guidi, A. Michienzi, L. Ricci, Evaluating the decentralisation of filecoin, in:Proceedings of the 3rd International Workshop on Distributed Infrastructure for the Common Good, 2022, pp. 13-18.

[78] Protocol Labs, Lotus, 2023, https://lotus.filecoin.io/storage-providers/get-started/hardware-requirements/.

[79] Y. Guo, M. Xu, D. Yu, Y. Yu, R. Ranjan, X. Cheng, Cross-channel: Scalable off-chain channels supporting fair and atomic cross-chain operations, IEEE Trans. Comput. (2023).

[80] X. Shi, Q. Li, D. Wang, L. Lu, Mobile Computing Force Network (MCFN): Computing and network convergence supporting integrated communication service,in:2022 International Conference on Service Science, ICSS, 2022, pp. 131-136.

[81] S. de Figueiredo, A. Madhusudan, V. Reniers, S. Nikova, B. Preneel, Exploring the storj network: A security analysis,in:Proceedings of the 36th Annual ACM Symposium on Applied Computing, 2021, pp. 257-264.

[82] X. Wang, S. Azouvi, M. Vukolić, Security analysis of filecoin’s expected consensus in the Byzantine vs honest model, 2023, arXiv preprint arXiv: 2308.06955.

[83] T. Cao, X. Li, Temporary block withholding attacks on filecoin’s expected consensus, in:Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, 2023, pp. 109-122.

[84] C. Li, B. Palanisamy, R. Xu, L. Duan, J. Liu, W. Wang, How hard is takeover in DPoS blockchains? Understanding the security of coin-based voting governance, in:Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023, pp. 150-164.

[85] R. Matzutt, J. Hiller, M. Henze, J.H. Ziegeldorf, D. Müllmann, O. Hohlfeld, K. Wehrle, A quantitative analysis of the impact of arbitrary blockchain content on bitcoin, in: Financial Cryptography and Data Security: 22nd International Conference, FC 2018, Nieuwpoort, CuraÇAo, February 26-March 2, 2018, Revised Selected Papers 22, Springer, 2018, pp. 420-438.

[86] S. Namane, I. Ben Dhaou, Blockchain-based access control techniques for iot applications, Electronics 11 (14) (2022) 2225.

[87] C. Liu, M. Xu, H. Guo, X. Cheng, Y. Xiao, D. Yu, B. Gong, A. Yerukhimovich, S. Wang, W. Lyu, TBAC: A Tokoin-based accountable access control scheme for the internet of things, IEEE Trans. Mob. Comput. (2023) 1-16.

[88] A. Chatterjee, Y. Pitroda, M. Parmar,Dynamic role-based access control for decentralized applications, in:Blockchain-ICBC 2020: Third International Conference, Held As Part of the Services Conference Federation, SCF 2020, Honolulu, HI, USA, September 18-20, 2020, Proceedings 3, Springer, 2020, pp. 185-197.