Energy-Efficient Underwater Data Collection: A Q-Learning Based Approach

Abstract

Underwater data collection is an importance part in the process of network monitoring, network management and intrusion detection. However, the limited-energy of nodes is a major challenge to collect underwater data. The solution of this problem are not only in the hands of network topology but in the hands of path of autonomous underwater vehicle (AUV). With the problem in hand, an energy-efficient data collection scheme is designed for mobile underwater network. Especially, the data collection scheme is divided into two phases, i.e., routing algorithm design for sensor nodes and path planing for AUV. With consideration of limited-energy and network robustness, Q-learning based dynamic routing algorithm is designed in the first phase to optimize the routing selection of nodes, through which a potential-game based optimal rigid graph method is proposed to balance the trade-off between the energy consumption and the network robustness. With the collected data, Q-learning based path planning strategy is proposed for AUV in the second phase to find the desired path to gather the data from data collector, then a mode-free tracking controller is developed to track the desired path accurately. Finally, the performance analysis and simulation results reveal that the proposed approach can guarantee energy-efficient and improve network stability.

Keywords

Underwater data collection / Q-learning / Energy efficient / Path planning / Autonomous underwater vehicle

References

[1] Ali T, Jung L, Faye I. End-to-end delay and energy efficient routing protocol for underwater wireless sensor networks. Wireless Personal Communications, 2014, 79(1): 339-361

[2] Bhadauria D, Tekdas O, Isler V. Robotic data mules for collecting data over sparse sensor fields. Journal of Field Robotics, 2011, 28(3): 388-404

[3] Caccia M, Veruggio G. Guidance and control of a reconfigurable unmanned underwater vehicle. Control Engineering Practice, 2000, 8(1): 21-37

[4] Cai S, Zhu Y, Wang T, Xu G, Liu A, Liu X. Data collection in underwater sensor networks based on mobile edge computing. IEEEAccess, 2019, 7: 65357-65367

[5] Cashmore M, Fox M, Long D, Magazzeni D, Ridder B. Opportunistic planning in autonomous underwater missions. IEEE Transactions on Automation Science and Engineering, 2018, 15(2): 519-530

[6] Chen J, Xu Y, Wu Q, Zhang Y, Chen X, Qi N. Interference-aware online distributed channel selection for multicluster fanet: A potential game approach. IEEE Transactions on Vehicular Technology, 2019, 68(4): 3792-3804

[7] Dhurandher S, Khairwal S, Obaidat M, Misra S. Efficient data acquisition in underwater wireless sensor ad hoc networks. IEEE Wireless Communications, 2009, 16(6): 70-78

[8] Duan R, Du J, Jiang C, Ren Y. Value-based hierarchical information collection for auv-enabled internet of underwater things. IEEE Internet of Things Journal, 2020, 7(10): 9870-9883

[9] Gjanci P, Petrioli C, Basagni S, Phillips C, Bölöni L, Turgut D. Path finding for maximum value of information in multimodal underwater wireless sensor networks. IEEE Transactions on Mobile Computing, 2018, 17(2): 404-418

[10] Gong Z, Li C, Jiang F. A machine learning-based approach for auto-detection and localization of targets in underwater acoustic array networks. IEEE Transactions on Vehicular Technology, 2020, 69(12): 15857-15866

[11] Han G, Li S, Zhu C, Jiang J, Zhang W. Probabilistic neighborhood-based data collection algorithms for 3D underwater acoustic sensor networks. Sensors, 2017, 17(2): 1-16

[12] Han G, Shen S, Song H, Yang T, Zhang W. A stratification-based data collection scheme in underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology, 2018, 67(11): 10671-10682

[13] Han G, Shen S, Wang H, Jiang J, Guizani M. Prediction-based delay optimization data collection algorithm for underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology, 2019, 68(7): 6926-6936

[14] Jin Z, Ma Y, Su Y, Li S, Fu X. A Q-learning-based delay-aware routing algorithm to extend the lifetime of underwater sensor networks. Sensors, 2017, 17(7): 1-15

[15] Khan J, Cho H (2014) A data gathering protocol using AUV in underwater sensor networks. Proceedings of OCEANS, Taipei, 1–6. https://doi.org/10.3390/s150819331

[16] Khan J, Cho H. Data-gathering scheme using auvs in large-scale underwater sensor networks: A multihop approach. Sensors, 2016, 16(10): 1-20

[17] Kim J, Joe H, Yu S, Lee J, Kim M. Time-delay controller design for position control of autonomous underwater vehicle under disturbances. IEEE Transactions on Industrial Electronics, 2016, 63(2): 1052-1061

[18] Lee S, Park S, Choi H. Potential game-based non-myopic sensor network planning for multi-target tracking. IEEE Access, 2018, 6: 79245-79257

[19] Li C, Xu Y, Diao B, Wang Q, An Z. DBR-MAC: A depth-based routing aware mac protocol for data collection in underwater acoustic sensor networks. IEEE Sensors Journal, 2016, 16(10): 3904-3913

[20] Li G, Peng S, Wang C, Niu J, Yuan Y. An energy-efficient data collection scheme using denoising autoencoder in wireless sensor networks. Tsinghua Science and Technology, 2019, 24(1): 86-96

[21] Li N, Martínez J, Meneses Chaus J, Eckert M. A survey on underwater acoustic sensor network routing protocols. Sensors, 2016, 16(3): 1-28

[22] Lin C, Han G, Guizani M, Bi Y, Du J. A scheme for delay-sensitive spatiotemporal routing in sdn-enabled underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology, 2019, 68(9): 9280-9292

[23] Lin C, Deng D, Wang S. Extending the lifetime of dynamic underwater acoustic sensor networks using multi-population harmony search algorithm. IEEE Sensors Journal, 2016, 16(11): 4034-4042

[24] Liu Z, Meng X, Liu Y, Yang Y, Wang Y. AUV-aided hybrid data collection scheme based on value of information for internet of underwater things. IEEE Internet of Things Journal, 2022, 9(9): 6944-6955

[25] Luo X, Li X, Wang J, Guan X. Potential-game based optimally rigid topology control in wireless sensor networks. IEEE Access, 2018, 6: 16599-16609

[26] Luo X, Yan Y, Li S, Guan X. Topology control based on optimally rigid graph in wireless sensor networks. Computer Networks, 2013, 57(4): 1037-1047

[27] McMahon J, Plaku E. Autonomous data collection with limited time for underwater vehicles. IEEE Robotics and Automation Letters, 2017, 2(1): 112-119

[28] Ren J, Zhang Y, Zhang K, Liu A, Chen J, Shen X. Lifetime and energy hole evolution analysis in data-gathering wireless sensor networks. IEEE Transactions on Industrial Informatics, 2016, 12(2): 788-800

[29] Sehgal A, David C, Schönwälder J (2011) Energy consumption analysis of underwater acoustic sensor networks. Proceedings of OCEANS, Waikoloa, USA, 1–6. https://doi.org/10.23919/OCEANS.2011.6107287

[30] Shames I, Summers T. Rigid network design via submodular set function optimization. IEEE Transactions on Network Science and Engineering, 2015, 2(3): 84-96

[31] Su R, Zhang D, Li C, Gong Z, Venkatesan R, Jiang F. Localization and data collection in auv-aided underwater sensor networks: Challenges and opportunities. IEEE Network, 2019, 33(6): 86-93

[32] Thenozhi S, Yu W. Stability analysis of active vibration control of building structures using PD/PID control. Engineering Structures, 2014, 81: 208-218

[33] Wang Z, Song H, Watkins D, Ong K, Xue P, Yang Q, Shi X. Cyber-physical systems for water sustainability: challenges and opportunities. IEEE Communications Magazine, 2015, 53(5): 216-222

[34] Wei B, Luo Y, Jin Z, Wei J, Su Y (2013) ES-VBF: an energy saving routing protocol. Proceedings of the 2012 International Conference on Information Technology and Software Engineering, Beijing, 87–97. https://doi.org/10.1007/978-3-642-34528-9-10

[35] Wu J, Sun X, Wu J, Han G. Routing strategy of reducing energy consumption for underwater data collection. Intelligent and Converged Networks, 2021, 2(3): 163-176

[36] Xia N, Ou Y, Wang S, Zheng R, Du H, Xu C. Localizability judgment in uwsns based on skeleton and rigidity theory. IEEE Transactions on Mobile Computing, 2017, 16(4): 980-989

[37] Xie P, Cui J, Lao L. VBF: Vector-based forwarding protocol for underwater sensor networks, 2006, Berlin, Heidelberg: Springer, 1216-1221

[38] Yan J, Chen C, Luo X, Yang X, Hua C, Guan X. Distributed formation control for teleoperating cyber-physical system under time delay and actuator saturation constrains. Information Sciences, 2016, 370: 680-694

[39] Yan J, Zhao H, Meng Y, Guan X (2021a) Localization in underwater sensor networks. ser. Wireless Networks, Singapore Springer. https://doi.org/10.1007/978-981-16-4831-1

[40] Yan J, Yang X, Zhao H, Luo X, Guan X (2021b) Autonomous underwater vehicles: Localization, tracking, and formation. ser. Cognitive Intelligence and Robotics, Singapore Springer. https://doi.org/10.1007/978-981-16-6096-2

[41] Yan J, Yang X, Luo X, Chen C. Energy-efficient data collection over auv-assisted underwater acoustic sensor network. IEEE Systems Journal, 2018, 12(4): 3519-3530

[42] Yan J, Zhao H, Luo X, Wang Y, Chen C, Guan X. Asynchronous localization of underwater target using consensus-based unscented kalman filtering. IEEE Journal of Oceanic Engineering, 2020, 45(4): 1466-1481

[43] Yan J, Zhao H, Pu B, Luo X, Chen C, Guan X. Energy-efficient target tracking with uasns: A consensus-based bayesian approach. IEEE Transactions on Automation Science and Engineering, 2020, 17(3): 1361-1375

[44] Yu H, Yao N, Liu J. An adaptive routing protocol in underwater sparse acoustic sensor networks. Ad Hoc Networks, 2015, 34: 121-143

[45] Zhang P, de Queiroz M, Cai X. Three-dimensional dynamic formation control of multi-agent systems using rigid graphs. Journal of Dynamic Systems, Measurement, and Control, 2015, 137(11): 1-7

[46] Zhao C, Guo L. PID controller design for second order nonlinear uncertain systems. Science China Information Sciences, 2017, 60(2): 1-13