Deep Learning in Maritime Autonomous Surface Ships: Current Development and Challenges
Jun Ye , Chengxi Li , Weisong Wen , Ruiping Zhou , Vasso Reppa
Journal of Marine Science and Application ›› 2023, Vol. 22 ›› Issue (3) : 584 -601.
Deep Learning in Maritime Autonomous Surface Ships: Current Development and Challenges
Autonomous surface ships have become increasingly interesting for commercial maritime sectors. Before deep learning (DL) was proposed, surface ship autonomy was mostly model-based. The development of artificial intelligence (AI) has prompted new challenges in the maritime industry. A detailed literature study and examination of DL applications in autonomous surface ships are still missing. Thus, this article reviews the current progress and applications of DL in the field of ship autonomy. The history of different DL methods and their application in autonomous surface ships is briefly outlined. Then, the previously published works studying DL methods in ship autonomy have been categorized into four groups, i.e., control systems, ship navigation, monitoring system, and transportation and logistics. The state-of-the-art of this review paper majorly lies in presenting the existing limitations and innovations of different applications. Subsequently, the current issues and challenges for DL application in autonomous surface ships are discussed. In addition, we have proposed a comparative study of traditional and DL algorithms in ship autonomy and also provided the future research scope as well.
Maritime autonomous surface ships / Deep learning (DL) / Artificial intelligence (AI) / Review
| [1] |
|
| [2] |
|
| [3] |
|
| [4] |
Bentes C, Frost A, Velotto D, Tings B (2016) Ship-iceberg discrimination with convolutional neural networks in high resolution SAR images. In Proceedings of EUSAR 2016: 11th European Conference on Synthetic Aperture Radar, 1–4 |
| [5] |
Cao X, Shu G, Chen L, Wang Y (2020) Ship recognition method combined with image segmentation and deep learning feature extraction in video surveillance. Multimedia Tools and Applications, 79. https://doi.org/10.1007/s11042-018-7138-3 |
| [6] |
|
| [7] |
|
| [8] |
Chen X, Ghadirzadeh A, Folkesson J, Björkman M, Jensfelt P (2018) Deep reinforcement learning to acquire navigation skills for wheel-legged robots in complex environments. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3110–3116. https://doi.org/10.48550/arXiv.1804.10500 |
| [9] |
|
| [10] |
|
| [11] |
|
| [12] |
Crisp DJ (2004) The state-of-the-art in ship detection in synthetic aperture radar imagery. organic letters. https://doi.org/10.1002/chin.200442210 |
| [13] |
|
| [14] |
Dauphin Y, Pascanu R, Gulcehre C, Cho K, Ganguli S, Bengio Y (2014) Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. arXiv preprint arXiv: 1406.2572 |
| [15] |
|
| [16] |
|
| [17] |
|
| [18] |
|
| [19] |
|
| [20] |
|
| [21] |
|
| [22] |
|
| [23] |
|
| [24] |
|
| [25] |
|
| [26] |
|
| [27] |
|
| [28] |
|
| [29] |
Han Z, Chong J (2004) A review of ship detection algorithms in polarimetric sar images. In International Conference on Signal Processing. https://doi.org/10.1109/ICOSR2004.1442203 |
| [30] |
|
| [31] |
|
| [32] |
|
| [33] |
|
| [34] |
Howard IM (1994) A review of rolling element bearing vibration detection, diagnosis and prognosis. NASA STI/Recon Technical Report N |
| [35] |
|
| [36] |
|
| [37] |
IMO (2021) Imo takes first steps to address autonomous ships. http://www.imo.org/en/mediacentre/pressbriefings/pages/08-msc-99-mass-scoping.aspx. [Accessed May 4, 2021] |
| [38] |
IMO (2019) Interim guidelines for mass trials. https://wwwcdn.imo.org/localresources/en/MediaCentre/HotTopics/Documents/MSC.1-Circ.1604%20-%20Interim%20Guidelines%20For%20Mass%20Trials%20(Secretariat).pdf |
| [39] |
Inggs M, Robinson A (1995) Neural approaches to ship target recognition. In Proceedings International Radar Conference, 386–391. https://doi.org/10.1109/RADAR.1995.522577 |
| [40] |
|
| [41] |
|
| [42] |
|
| [43] |
|
| [44] |
|
| [45] |
Jurgen and Ronald K (2013) Autonomous vehicles for safer driving. SAE International |
| [46] |
Kang M, Leng X, Lin Z, Ji K (2017) A modified faster r-cnn based on cfar algorithm for sar ship detection. In 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP), 1–4. https://doi.org/10.1109/RSIP.2017.7958815 |
| [47] |
|
| [48] |
|
| [49] |
|
| [50] |
|
| [51] |
Lazarowska A (2019) Research on algorithms for autonomous navigation of ships. WMU Journal of Maritime Affairs, 18. https://doi.org/10.1007/s13437-019-00172-0 |
| [52] |
|
| [53] |
|
| [54] |
|
| [55] |
|
| [56] |
|
| [57] |
|
| [58] |
|
| [59] |
|
| [60] |
|
| [61] |
Lloyd’s Register (2015) Design code for unmanned marine systems. https://www.cdinfo.lr.org/information/documents/ShipRight/Design%20and%20Construction/Additional%20Design%20Procedures/Design%20Code%20for%20Unmanned%20Marine%20Systems/Design%20Code%20for%20Unmanned%20Marine%20Systems,%20February%202017.pdf. [Accessed February 4, 2021] |
| [62] |
|
| [63] |
|
| [64] |
|
| [65] |
Morillas JRA, García IC, Zölzer U (2015) Ship detection based on svm using color and texture features. In 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), 343–350. https://doi.org/10.1109/ICCP.2015.7312682 |
| [66] |
|
| [67] |
MUNIN (2021)About munin-maritime unmanned navigation through intelligence in networks. http://www.unmanned-ship.org/munin/about/ Accessed February 4, 2021 |
| [68] |
|
| [69] |
|
| [70] |
Noel A, Shreyanka K, Kumar K, Shameem BM, Akshar B (2019) Autonomous ship navigation methods: A review. In International Conference on Marine Engineering and Technology Oman. https://doi.org/10.24868/icmet.oman.2019.028 |
| [71] |
|
| [72] |
Perera LP (2018) Autonomous ship navigation under deep learning and the challenges in colregs. In International Conference on Offshore Mechanics and Arctic Engineering, volume Volume 11B: Honoring Symposium for Professor Carlos Guedes Soares on Marine Technology and Ocean Engineering. https://doi.org/10.1115/OMAE2018-77672 |
| [73] |
|
| [74] |
|
| [75] |
|
| [76] |
|
| [77] |
Rosenblatt F (1962) Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. Cornell Aeronautical Laboratory. https://doi.org/10.1001/archpsyc.1962.01720030064010 |
| [78] |
|
| [79] |
|
| [80] |
|
| [81] |
Schwegmann CP, Kleynhans W, Salmon BP, Mdakane LW, and Meyer RGV (2016). Very deep learning for ship discrimination in synthetic aperture radar imagery. In 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 104–107. https://doi.org/10.1109/IGARSS.2016.7729017 |
| [82] |
Senior A, Vanhoucke V, Nguyen P, Sainath T, et al. (2012). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine, https://doi.org/10.1109/MSP.2012.2205597 |
| [83] |
|
| [84] |
|
| [85] |
|
| [86] |
|
| [87] |
|
| [88] |
|
| [89] |
|
| [90] |
|
| [91] |
Sun W, Gao X (2021) Deep learning-based trajectory tracking control for unmanned surface vehicle. Mathematical Problems in Engineering, 2021. https://doi.org/10.1155/2021/8926738 |
| [92] |
Tuyen LP, Layek A, Vien NA, Chung T (2017) Deep reinforcement learning algorithms for steering an underactuated ship. In 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 602–607. https://doi.org/10.1109/MFI.2017.8170388 |
| [93] |
Vandeginste B, Massart D, Buydens L, De Jong S, Lewi P, Smeyers-Verbeke J (1998). Chapter 44-artificial neural networks. In Vandeginste B, Massart D, Buydens L, De Jong S, Lewi P, Smeyers-Verbeke J, edited. Handbook of Chemometrics and Qualimetrics: Part B, volume 20 of Data Handling in Science and Technology, 649–699. Elsevier |
| [94] |
|
| [95] |
|
| [96] |
|
| [97] |
|
| [98] |
Wang Y, Tong J, Song TY, Wan ZH (2018b) Unmanned surface vehicle course tracking control based on neural network and deep deterministic policy gradient algorithm. In 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO), 1–5. https://doi.org/10.1109/OCEANSKOBE.2018.8559329 |
| [99] |
|
| [100] |
Wang Y, Wang C, Zhang H, Dong Y, Wei S (2019a) Automatic ship detection based on retinanet using multi-resolution gaofen-3 imagery. Remote Sensing, 11(5). https://doi.org/10.3390/rs11050531 |
| [101] |
|
| [102] |
|
| [103] |
|
| [104] |
|
| [105] |
Wu DF, Yang GH (2014) Review on key techniques for ship dynamic positioning system. Ship Science and Technology |
| [106] |
|
| [107] |
|
| [108] |
|
| [109] |
|
| [110] |
|
| [111] |
Zhang Q, Pan W, Reppa V (2020b) Model-reference reinforcement learning control of autonomous surface vehicles. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 5291–5296, https://doi.org/10.1109/CDC42340.2020.9304347 |
| [112] |
|
| [113] |
|
| [114] |
|
| [115] |
|
| [116] |
|
| [117] |
|
| [118] |
|
| [119] |
|
| [120] |
|
/
| 〈 |
|
〉 |