Automated Ship Berthing Guidance Method Based on Three-dimensional Target Measurement
Yiming Ma , Chao Mi , Lei Yao , Yi Liu , Weijian Mi
Journal of Marine Science and Application ›› 2023, Vol. 22 ›› Issue (2) : 172 -180.
Automated Ship Berthing Guidance Method Based on Three-dimensional Target Measurement
Automatic berthing guidance is an important aspect of automated ship technology to obtain the ship-shore position relationship. The current mainstream measurement methods for ship-shore position relationships are based on radar, multisensor fusion, and visual detection technologies. This paper proposes an automated ship berthing guidance method based on three-dimensional (3D) target measurement and compares it with a single-target recognition method using a binocular camera. An improved deep object pose estimation (DOPE) network is used in this method to predict the pixel coordinates of the two-dimensional (2D) keypoints of the shore target in the image. The pixel coordinates are then converted into 3D coordinates through the camera imaging principle, and an algorithm for calculating the relationship between the ship and the shore is proposed. Experiments were conducted on the improved DOPE network and the actual ship guidance performance to verify the effectiveness of the method. Results show that the proposed method with a monocular camera has high stability and accuracy and can meet the requirements of automatic berthing.
Automated ship / Automatic berthing / Berthing guidance / 3D measurement / Neural networks / Deep learning / Position estimation
| [1] |
Akbar J, Shahzad M, Malik MI, Ul-Hasan A, Shafait F (2019) Runway detection and localization in aerial images using deep learning. In 2019 IEEE Digital Image Computing: Techniques and Applications (DICTA), 1–8. https://doi.org/10.1109/DICTA47822.2019.8945889 |
| [2] |
Chang JR, Chen YS (2018) Pyramid stereo matching network. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5410–5418. https://doi.org/10.1109/CVPR.2018.00567 |
| [3] |
Cheng Y, Li B (2021) Image segmentation technology and its application in digital image processing. In 2021 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), 1174–1177. https://doi.org/10.1109/IPEC51340.2021.9421206 |
| [4] |
|
| [5] |
Dwina N, Arnia F, Munadi K (2018) Skin segmentation based on improved thresholding method. In 2018 IEEE International ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI-NCON), 95–99. https://doi.org/10.1109/ECTI-NCON.2018.8378289 |
| [6] |
Fu MY, Xu YJ, Wang YH (2015) Cooperation and collision avoidance for multiple DP ships with disturbances. In 2015 IEEE 34th Chinese Control Conference (CCC), 4208–4213. https://doi.org/10.1109/ChiCC.2015.7260288 |
| [7] |
|
| [8] |
|
| [9] |
|
| [10] |
|
| [11] |
Li TY, Zhu HB (2018) Research on model control of binocular robot vision system. In 2018 IEEE Chinese Automation Congress (CAC), 1794–1797. https://doi.org/10.1109/CAC.2018.8623756 |
| [12] |
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: Single shot multibox detector. In Proceedings of Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, Part I, 21–37. Springer International Publishing. https://doi.org/10.48550/arXiv.1512.02325 |
| [13] |
|
| [14] |
|
| [15] |
Morrical N, Tremblay J, Lin Y, Tyree S, Birchfield S, Pascucci V, Wald I (2021) NViSII: A scriptable tool for photorealistic image generation. arXiv preprint arXiv: 2105.13962. https://doi.org/10.48550/arXiv.2105.13962 |
| [16] |
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779–788. https://doi.org/10.1109/CVPR.2016.91 |
| [17] |
|
| [18] |
Shuo J, Yonghui Z, Wen R, Kebin T (2017) The unmanned autonomous cruise ship for water quality monitoring and sampling. In 2017 IEEE International Conference on Computer Systems, Electronics and Control (ICCSEC), 700–703. https://doi.org/10.1109/ICCSEC.2017.8447040 |
| [19] |
Song Q, Lin GY, Ma JQ, Zhang HM (2016) An edge-detection method based on adaptive canny algorithm and iterative segmentation threshold. In IEEE 2016 2nd International Conference on Control Science and Systems Engineering (ICCSSE), 64–67. https://doi.org/10.1109/CCSSE.2016.7784354 |
| [20] |
Sun ZY, Wang LH, Liu LQ (2020) Three-dimensional reconstruction algorithm based on inverse perspective transformation. In 2020 IEEE International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), 221–225. https://doi.org/10.1109/ICBAIE49996.2020.00053 |
| [21] |
Tateno K, Tombari F, Laina I, Navab N (2017) CNN-SLAM: Real-time dense monocular slam with learned depth prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6243–6252. https://doi.org/10.1109/CVPR.2017.695 |
| [22] |
Tekin B, Sinha SN, Fua P (2018) Real-time seamless single shot 6d object pose prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, 292–301. https://doi.org/10.1109/CVPR.2018.00038 |
| [23] |
Tremblay J, To T, Sundaralingam B, Xiang Y, Fox D, Birchfield S (2018) Deep object pose estimation for semantic robotic grasping of household objects. arXiv preprint arXiv: 1809.10790. https://doi.org/10.48550/arXiv.1809.10790 |
| [24] |
Xiang Y, Schmidt T, Narayanan V, Fox D (2017) Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv: 1711.00199. https://doi.org/10.48550/arXiv.1711.00199 |
| [25] |
|
| [26] |
Zhao W, Zhang S, Guan Z, Zhao W, Peng J, Fan J (2020) Learning deep network for detecting 3d object keypoints and 6d poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14134–14142. https://doi.org/10.1109/CVPR42600.2020.01414 |
/
| 〈 |
|
〉 |