A new technology perspective of the Metaverse: Its essence, framework and challenges

Abstract

The Metaverse depicts a parallel digitalized world where virtuality and reality are fused. It has economic and social systems like those in the real world and provides intelligent services and applications. In this paper, we introduce the Metaverse from a new technology perspective, including its essence, corresponding technical framework, and potential technical challenges. Specifically, we analyze the essence of the Metaverse from its etymology and point out breakthroughs promising to be made in time, space, and contents of the Metaverse by citing Maslow's Hierarchy of Needs. Subsequently, we conclude four pillars of the Metaverse, named ubiquitous connections, space convergence, virtuality and reality interaction, and human-centered communication, and establish a corresponding technical framework. Additionally, we envision open issues and challenges of the Metaverse in the technical aspect. The work proposes a new technology perspective of the Metaverse and will provide further guidance for its technology development in the future.

Keywords

Metaverse / Technical framework / Ubiquitous connections / Space convergence / Virtuality and reality interaction / Human-centered communication

References

[1] H. Ning, X. Ye, M.A. Bouras, D. Wei, M. Daneshmand, General cyberspace: cyberspace and cyber-enabled spaces, IEEE Internet Things J. 5 (3) (2018) 1843-1856.

[2] H. Ning, H. Wang, Y. Lin, W. Wang, S. Dhelim, F. Farha, J. Ding, M. Daneshmand, A Survey on Metaverse: the State-Of-The-Art, Technologies, Applications, and Challenges, IEEE Internet of Things J. 10 (16) (2023) 14671-14688.

[3] C. Jaynes, W.B. Seales, K. Calvert, Z. Fei, J. Griffioen,The metaverse: a networked collection of inexpensive, self-configuring, immersive environments,in:Proceedings of the Workshop on Virtual Environments 2003, 2003, pp. 115-124.

[4] S. Spielberg, A. Silvestri, Z. Penn, et al., Ready Player One, Warner Bros USA, 2018.

[5] H. Duan, J. Li, S. Fan, Z. Lin, X. Wu, W. Cai, Metaverse for social good: a university campus prototype,in: Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 153-161.

[6] R. Smartsearch, Something about the six core technologies in the metaverse. https://ramaonhealthcare.com/something-about-the-six-core-technologies-in-the-metaverse/. (Accessed 30 September 2022).

[7] L. Lee, T. Braud, P. Zhou, L. Wang, D. Xu, Z. Lin, A. Kumar, C. Bermejo, P. Hui, All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda, Foundations and Trends® in Human-Computer Interaction 18 ( 2-3) (2024) 100-337.

[8] J.D.N. Dionisio, W.G. B III, R. Gilbert, 3d virtual worlds and the metaverse: current status and future possibilities, ACM Comput. Surv. 45 (3) (2013) 1-38.

[9] F.A. Wolf, Parallel Universes, Simon and Schuster, 1988.

[10] S. McLeod, Maslow's hierarchy of needs, Simply psychol. 1 (2007) 1-16.

[11] W. Chris, Ubiquitous connectivity is here, and it changes everything. https://www.forbes.com/sites/forbestechcouncil/2021/08/25/ubiquitous-connectivity-is-here-and-it-changes-everything/?sh=1361147e7cb1. (Accessed 30 November 2021).

[12] H. Ning, Unit and Ubiquitous Internet of Things, CRC press, 2013.

[13] F.M. Schaf, S. Paladini, C.E. Pereira, 3d autosyslab prototype, in: Proceedings of the 2012 IEEE Global Engineering Education Conference, IEEE, 2012, pp. 1-9.

[14] Y. Han, D. Niyato, C. Leung, C. Miao, D.I. Kim, A Dynamic Resource Allocation Framework for Synchronizing Metaverse with Iot Service and Data, ICC 2022, IEEE International Conference on Communications, IEEE, 2022, pp. 1196-1201.

[15] T. Ruohom€aki, E. Airaksinen, P. Huuska, O. Kes€aniemi, M. Martikka, J. Suomisto, Smart city platform enabling digital twin, in: 2018 International Conference on Intelligent Systems, IEEE, 2018, pp. 155-161.

[16] Q. Lu, A.K. Parlikad, P. Woodall, et al., Developing a digital twin at building and city levels: case study of west cambridge campus, J. Manag. Eng. 36 (3) (2020) 05020004.

[17] R. Baheti, H. Gill, Cyber Phys. Syst. Impact Control Technol. 12 (1) (2011) 161-166.

[18] Y. Zhou, F.R. Yu, J. Chen, Y. Kuo, Cyber-physical-social systems: a state-of-the-art survey, challenges and opportunities, IEEE Commun. Survey Tutorial. 22 (1)(2019) 389-425.

[19] A. Khanna, R. Anand, Iot based smart parking system, in: 2016 International Conference on Internet of Things and Applications (IOTA), IEEE, 2016, pp. 266-270.

[20] F. Tao, Q. Qi, L. Wang, A. Nee,Digital twins and cyber-physical systems toward smart manufacturing and industry 4.0: correlation and comparison, Engineering 5 (4) (2019) 653-661.

[21] P.A. Grabowicz, J.J. Ramasco, E. Moro, J.M. Pujol, et al., Social features of online networks: the strength of intermediary ties in online social media, PLoS One 7 (1)(2012) 29358-29366.

[22] S. Steinert, O. Friedrich, Wired emotions: ethical issues of affective brain-computer interfaces, Sci. Eng. Ethics 26 (1) (2020) 351-367.

[23] E. Musk, et al., An integrated brain-machine interface platform with thousands of channels, J. Med. Internet Res. 21 (10) (2019) e16194.

[24] T. Lewis, Elon musk’s pig-brain implant is still a long way from ‘solving paralysis’. https://www.scientificamerican.com/article/elon-musks-pig-brain-implant-is-still-a-long-way-from-solving-paralysis/. (Accessed 30 November 2021).

[25] X. Hu, R. Su, L. He, The design and implementation of the 3d educational game based on vr headsets, in: 2016 International Symposium on Educational Technology, IEEE, 2016, pp. 53-56.

[26] R. Epp, D. Lin, C. Bezemer, An empirical study of trends of popular virtual reality games and their complaints, IEEE Trans. Game. 13 (3) (2021) 275-286.

[27] Y. Liu, S. Wu, Q. Xu, H. Liu, Holographic projection technology in the field of digital media art, 2021, Wireless Commun. Mobile Comput. (2021) 1-12.

[28] Z. Guo, Z. Wang, X. Jin, Avatar to person”(atp) virtual human social ability enhanced system for disabled people, 2021, Wireless Commun. Mobile Comput.(2021) 1-10.

[29] W. Chang, H. Shin, Virtual experience in the performing arts: K-live hologram music concerts, Popular Entertainment Stud. 10 (1-2) (2020) 34-50.

[30] G. Sagl, B. Resch, Mobile Phones as Ubiquitous Social and Environmental Geo-Sensors, IGI Global, 2015.

[31] Q. Li, W. Huangfu, F. Farha, T. Zhu, S. Yang, L. Chen, H. Ning, Multi-resident type recognition based on ambient sensors activity, Future Generat. Comput. Syst. 112 (2020) 108-115.

[32] K. Chang, Bluetooth: a viable solution for iot?[industry perspectives], IEEE Wireless Commun. 21 (6) (2014) 6-7.

[33] J. Yang, H. Zou, H. Jiang, L. Xie, Device-free occupant activity sensing using wifi-enabled iot devices for smart homes, IEEE Internet Things J. 5 (5) (2018) 3991-4002.

[34] J. Huang, F. Ruan, M. Su, X. Yang, S. Yao, J. Zhang,Analysis of orthogonal frequency division multiplexing (ofdm) technology in wireless communication process, in: 2016 10th IEEE International Conference on Anti-counterfeiting, 2016, pp. 122-125. Security, and Identification, IEEE.

[35] L. Chettri, R. Bera, A comprehensive survey on internet of things (iot) toward 5g wireless systems, IEEE Internet Things J. 7 (1) (2019) 16-32.

[36] D.C. Nguyen, M. Ding, P.N. Pathirana, et al., 6g internet of things: a comprehensive survey, IEEE Internet Things J. 9 (1) (2021) 359-383.

[37] P.H. Winston, Artificial Intelligence, Addison-Wesley Longman Publishing Co., Inc., 1992.

[38] M.I. Jordan, T.M. Mitchell, Machine learning: trends, perspectives, and prospects, Science 349 (6245) (2015) 255-260.

[39] B. Mahesh, Machine learning algorithms-a review, Int. J. Sci. Res. 9 (2020) 381-386.

[40] J. Pujara, H. Miao, L. Getoor, W. Cohen, Knowledge graph identification, in: International Semantic Web Conference, Springer, 2013, pp. 542-557.

[41] J. Hirschberg, C.D. Manning, Advances in natural language processing, Science 349 (6245) (2015) 261-266.

[42] C. Wu, et al., Natural language processing for smart construction: current status and future directions, Autom. ConStruct. 134 (2022) 104059-104080.

[43] A.V. Haridas, R. Marimuthu, V.G. Sivakumar, A critical review and analysis on techniques of speech recognition: the road ahead, Int. J. Knowl. Base. Intell. Eng. Syst. 22 (1) (2018) 39-57.

[44] X. Feng, Y. Jiang, X. Yang, et al., Computer vision algorithms and hardware implementations: a survey, Integration 69 (2019) 309-320.

[45] E. Soegoto, R. Utami, Y. Hermawan, Influence of artificial intelligence in automotive industry, in: Journal of Physics: Conference Series, IOP Publishing, 2019, pp. 66081-66086.

[46] T. Huynhthe, Q. Pham, X. Pham, et al., Artificial Intelligence for the Metaverse: A Survey, Engineering Applications of Artificial Intelligence, 117(2023)105581.

[47] W. Viriyasitavat, D. Hoonsopon, Blockchain characteristics and consensus in modern business processes, J. Indust. Info Integrate. 13 (2019) 32-39.

[48] V. Chang, P. Baudier, H. Zhang, et al., How blockchain can impact financial services-the overview, challenges and recommendations from expert interviewees, Technol. Forecast. Soc. Change 158 (2020) 120166-120176.

[49] E. Tijan, S. Aksentijević, K. Ivanić, M. Jardas, Blockchain technology implementation in logistics, Sustainability 11 (4) (2019) 1185-1192.

[50] M. Andoni, V. Robu, D. Flynn, S. Abram, D. Geach, D. Jenkins, P. McCallum, A. Peacock, Blockchain technology in the energy sector: a systematic review of challenges and opportunities, Renew. Sustain. Energy Rev. 100 (2019) 143-174.

[51] H. Jeon, H. Youn, S. Ko, T. Kim, Blockchain and ai meet in the metaverse, 10.5772, Adv. Convergence Blockchain Artificial Intelligence. 73 (2022) 73-82.

[52] T. Berners-Lee, R. Fielding, L. Masinter, Uniform resource identifier (uri): generic syntax, Tech. rep. (2005).

[53] D.L. Brock, The Electronic Product Code (Epc), Auto-ID Center White Paper MIT-AUTOID-WH-002, 2001, pp. 1-21.

[54] H. Ning, S. Hu, W. He, Q. Xu, H. Liu, W. Chen, nid-based internet of things and its application in airport aviation risk management, Chin. J. Electron. 21 (2) (2012) 209-214.

[55] H. Ning, Y. Fu, S. Hu, H. Liu, Tree-code modeling and addressing for non-id physical objects in the internet of things, Telecommun. Syst. 58 (3) (2015) 195-204.

[56] I. Szilagyi, P. Wira,Ontologies and semantic web for the internet of things-a survey, in: IECON 2016-42 nd Annual Conference of the IEEE Industrial Electronics Society, IEEE, 2016, pp. 6949-6954.

[57] H. Hasemann, A. Kr€oller,M. Pagel, The Wiselib Tuplestore: a Modular Rdf Database for the Internet of Things, arXiv preprint arXiv:1402.7228.

[58] W. Li, H. Leung, Y. Zhou, Space-time registration of radar and esm using unscented kalman filter, IEEE Trans. Aero. Electron. Syst. 40 (3) (2004) 824-836.

[59] S. Zhou, W. Cai, B.-S. Lee, S.J. Turner, Time-space consistency in large-scale distributed virtual environments, ACM Trans. Model Comput. Simulat 14 (1) (2004) 31-47.

[60] D. Zhong, S. Chang, Long-term moving object segmentation and tracking using spatio-temporal consistency, in: Proceedings 2001 International Conference on Image Processing, IEEE, 2001, pp. 57-60.

[61] P. Johnston,Authentication and Session Management on the Web, 2009. Retrieved. (Accessed 13 December 2004).

[62] K. Gutzmann, Access control and session management in the http environment, IEEE Internet Comput. 5 (1) (2001) 26-35.

[63] N. Poggi, T. Moreno, J.L. Berral, R. Gavalda, J. Torres, Self-adaptive utility-based web session management, Comput. Network. 53 (10) (2009) 1712-1721.

[64] K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, S. Tuecke, A resource management architecture for metacomputing systems, in: Workshop on Job Scheduling Strategies for Parallel Processing, Springer, 1998, pp. 62-82.

[65] H. Kim, J. Park, Y. Jeong, Efficient resource management scheme for storage processing in cloud infrastructure with internet of things, Wireless Pers. Commun. 91 (4) (2016) 1635-1651.

[66] D. Zhang, Y. Qiao, L. She, R. Shen, J. Ren, Y. Zhang, Two time-scale resource management for green internet of things networks, IEEE Internet Things J. 6 (1)(2018) 545-556.

[67] H. Yang, W. Zhong, C. Chen, A. Alphones, X. Xie, Deep-reinforcement-learning-based energy-efficient resource management for social and cognitive internet of things, IEEE Internet Things J. 7 (6) (2020) 5677-5689.

[68] J. Lanier, A. Heilbrun, A portrait of the young visionary, Whole Earth Rev. (1989) 108-119.

[69] G.C. Burdea, P. Coiffet, Virtual Reality Technology, John Wiley & Sons, 2003.

[70] D. Edler, O. Kühne, J. Keil, F. Dickmann, Audiovisual cartography: established and new multimedia approaches to represent soundscapes, KN J. Cartograph Geograph. Info. 69 (1) (2019) 5-17.

[71] M. Chang, J. Lambert, P. Sangkloy, J. Singh, et al., Argoverse: 3d tracking and forecasting with rich maps,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8748-8757.

[72] T. Li, W. Wang, Using wearable devices to participate in 3d interactive storytelling, in: International Conference on Interactive Digital Storytelling, Springer, 2021, pp. 80-93.

[73] W. Mphepo, Stereoscopy and Autostereoscopy, Mixed Reality, 2020.

[74] R. Silva, J.C. Oliveira, G.A. Giraldi, Introduction to augmented reality, Nat. Lab. Sci. Comput. 11 (2003) 1-11.

[75] A. Iatsyshyn, K. Valeriia, Y.O. Romanenko, et al., Application of augmented reality technologies for preparation of specialists of new technological era,in: 2nd International Workshop on Augmented Reality in Education, 2019.

[76] D. Tome, P. Peluse, L. Agapito, H. Badino, xr-egopose: egocentric 3d human pose from an hmd camera,in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7728-7738.

[77] S. Rokhsaritalemi, A. Sadeghiniaraki, S. Choi, A review on mixed reality: current trends, challenges and prospects, Appl. Sci. 10 (2) (2020) 636.

[78] A. Rezeika, M. Benda, P. Stawicki, F. Gembler, A. Saboor, I. Volosyak, Brain-computer interface spellers: a review, Brain Sci. 8 (4) (2018) 57.

[79] D. Marshall, D. Coyle, S. Wilson, M. Callaghan, Games, gameplay, and bci: the state of the art, IEEE Trans. Comput. Intelligence AI Game. 5 (2) (2013) 82-99.

[80] J. Gregory, Game Engine Architecture, AK Peters/CRC Press, 2018.

[81] R. Salama, M. ElSayed, Basic elements and characteristics of game engine, Global Comput. Sci. : Theory Res. 8 (3) (2018) 126-131.

[82] C. Fink,A new 3d game engine that means business. https://www.forbes.com/sites/charliefink/2020/05/28/a-new-3d-game-engine-that-means-business/?sh=68bd4b9a7530. (Accessed 30 January 2022).

[83] D.A.L. Carvajal, M.M. Morita, G.M. Bilmes, Virtual museums. captured reality and 3d modeling, J. Cult. Herit. 45 (2020) 234-239.

[84] R. Winzenrieth, L. Humbert, S. Di Gregorio, E. Bonel, M. García, L. Del Rio, Effects of osteoporosis drug treatments on cortical and trabecular bone in the femur using dxa-based 3d modeling, Osteoporos. Int. 29 (10) (2018) 2323-2333.

[85] M. Burgos, E. Sanmiguel-Rojas, N. Singh, F. Esteban-Ortega, Digbody®: a new 3d modeling tool for nasal virtual surgery, Comput. Biol. Med. 98 (2018) 118-125.

[86] T. Akeninemoller, E. Haines, N. Hoffman, Real-time Rendering, CRC Press, 2019.

[87] V. Sanzharov, V.A. Frolov, V.A. Galaktionov, 4, in: Survey of nvidia rtx technology, Programming and Computer Software, 46, 2020, pp. 297-304.

[88] S. Mystakidis, Metaverse, Encyclopedia 2 (1) (2022) 486-497.

[89] A.T. Maereg, A. Nagar, D. Reid, E.L. Secco, Wearable vibrotactile haptic device for stiffness discrimination during virtual interactions, Front Robot. AI 4 (2017) 42.

[90] X. Jin, Y. Yang, F. Jiang, J. Shuyuan, Social network structure feature analysis and its modelling [j], Bull. Chin. Acad. Sci. 30 (2) (2015) 216-228.

[91] M.A. Smith, B. Shneiderman, N. Milic-Frayling, E. Mendes Rodrigues, V. Barash, C. Dunne, T. Capone, A. Perer, E. Gleave, Analyzing (social media) networks with nodexl,in: Proceedings of the Fourth International Conference on Communities and Technologies, 2009, pp. 255-264.

[92] D.J. Watts, S.H. Strogatz, Collective dynamics of ‘small-world’networks, Nature 393 (6684) (1998) 440-442.

[93] A. Barabási, R. Albert, Emergence of scaling in random networks, Science 286 (5439) (1999) 509-512.

[94] A. Mislove, M. Marcon, K.P. Gummadi, P. Druschel, B. Bhattacharjee,Measurement and analysis of online social networks, in:Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, 2007, pp. 29-42.

[95] W. Fan, Y. Ma, Q. Li, Y. He, et al., Graph neural networks for social recommendation,in: The World Wide Web Conference, 2019, pp. 417-426.

[96] L. Luceri, T. Braun, S. Giordano, Social influence (deep) learning for human behavior prediction,in: International Workshop on Complex Networks, Springer, 2018, pp. 261-269.

[97] J. Qiu, J. Tang, H. Ma, Y. Dong, et al., Deepinf: social influence prediction with deep learning,in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 2110-2119.

[98] Z. Yu, X. Zhou, Socially Aware Computing: Concepts, Technologies, and Practices, Springer New York, New York, NY, 2014, pp. 9-23.

[99] B. Hull, V. Bychkovsky, Y. Zhang, K. Chen, M. Goraczko, A. Miu, E. Shih, H. Balakrishnan, S. Madden, Cartel: a distributed mobile sensor computing system,in: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems, 2006, pp. 125-138.

[100] Y. Liu, K. Wang, H. Guo, Q. Lu, Y. Sun, Social-aware computing based congestion control in delay tolerant networks, Mobile Network. Appl. 22 (2) (2017) 174-185.

[101] K. Zhang, J. Cao, H. Liu, S. Maharjan, Y. Zhang, Deep reinforcement learning for social-aware edge computing and caching in urban informatics, IEEE Trans. Ind. Inf. 16 (8) (2020) 5467-5477.

[102] F.F. Petiwala, V.K. Shukla, S. Vyas, Ibm watson: redefining artificial intelligence through cognitive computing,in:Proceedings of International Conference on Machine Intelligence and Data Science Applications, Springer, 2021, pp. 173-185.

[103] A. Sharma, A. Sharma, J.K. Pandey, M. Ram, Swarm Intelligence: Foundation, Principles, and Engineering Applications, CRC Press, 2022.

[104] G. Beni, J. Wang, Swarm intelligence in cellular robotic systems, in: Robots and Biological Systems: towards a New Bionics?, Springer, 1993, pp. 703-712.

[105] S. Luo, L. Cheng, B. Ren, Practical swarm optimization based fault-tolerance algorithm for the internet of things, KSII Trans. Internet Info Syst. TIIS 8 (3) (2014) 735-748.

[106] O. Zedadra, A. Guerrieri, N. Jouandeau, G. Spezzano, H. Seridi, G. Fortino, Swarm intelligence-based algorithms within iot-based systems: a review, J. Parallel Distr. Comput. 122 (2018) 173-187.

[107] J. Garcíanieto, E. Alba, A.C. Olivera, Swarm intelligence for traffic light scheduling: application to real urban areas, Eng. Appl. Artif. Intell. 25 (2) (2012) 274-283.

[108] U. Baum€ol, R. Jung, B.J. Kr€amer, Advances in Collective Intelligence and Social Media, 2013.