Underwater object detection and datasets: a survey

Abstract

The rapidly growing exploitation and utilization of marine resources by humans has sparked considerable interest in underwater object detection tasks. Targets captured in underwater environments differ significantly from those captured in general images owing to various factors, such as water turbidity, complex background conditions, and lighting variations. These adverse factors pose a host of challenges, such as high intensity noise, texture distortion, uneven illumination, low contrast, and limited visibility in underwater images. To address the specific difficulties encountered in underwater environments, numerous underwater object detection methods have been developed in recent years in response to these challenges. Furthermore, there has been a significant effort in constructing diverse and comprehensive underwater datasets to facilitate the development and evaluation of these methods. This paper outlines 14 traditional methods used in underwater object detection based on three aspects that rely on handmade features. Thirty-four more advanced technologies based on deep learning were presented from eight aspects. Moreover, this paper conducts a comprehensive study of seven representative datasets used in underwater object detection missions. Subsequently, the challenges encountered in current underwater object detection tasks were analyzed from five directions. Based on the findings, potential research directions are expected to promote further progress in this field and beyond.

Keywords

Underwater images / Object detection / Underwater dataset / Marine internet of things

References

[1] Bartyzel K. Adaptive Kuwahara filter. Signal Image Video Proc, 2016, 10: 663-670,

[2] Beijbom O, Edmunds PJ, Kline DI, Mitchell BG, Kriegman D (2012) Automated annotation of coral reef survey images. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, pp 1170–1177. https://doi.org/10.1109/CVPR.2012.6247798

[3] Bochkovskiy A, Wang CY, Liao HYM (2020) YOLOv4: optimal speed and accuracy of object detection. Preprint at arXiv: 2004.10934

[4] Cai BL, Xu XM, Jia K, Qing CM, Tao DC. DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Proc, 2016, 25(11): 5187-5198,

[5] Cao X, Zhang XM, Yu Y, Niu LT (2016) Deep learning-based recognition of underwater target. In: 2016 IEEE International Conference on Digital Signal Processing, Beijing, pp 89–93. https://doi.org/10.1109/ICDSP.2016.7868522

[6] Chen L, Liu ZH, Tong L, Jiang ZH, Wang SK, Dong JY et al (2020a) Underwater object detection using Invert Multi-Class Adaboost with deep learning. In: 2020 International Joint Conference on Neural Networks, Glasgow, pp 1–8. https://doi.org/10.1109/IJCNN48605.2020.9207506

[7] Chen L, Tong L, Zhou FX, Jiang ZH, Li ZY, Lv JL et al (2020b) A benchmark dataset for both underwater image enhancement and underwater object detection. Preprint at arXiv:2006.15789

[8] Chen L, Yang YY, Wang ZH, Zhang J, Zhou SW, Wu LH. Underwater target detection lightweight algorithm based on multi-scale feature fusion. J Mar Sci Eng, 2023, 11(2): 320,

[9] Chen X, Chen HJ (2010) A novel color edge detection algorithm in RGB color space. In: IEEE 10th International Conference on Signal Processing Proceedings, Beijing, pp 793–796. https://doi.org/10.1109/ICOSP.2010.5655926

[10] Chen Z, Gao HM, Zhang Z, Zhou HL, Wang X, Tian Y. Underwater salient object detection by combining 2D and 3D visual features. Neurocomputing, 2020, 391: 249-259,

[11] Chen Z, Zhang Z, Dai FZ, Bu Y, Wang HB. Monocular Vision-Based Underwater Object Detection. Sensors, 2017, 17(8): 1784, pmcid: 5580077

[12] Chen ZY, Zhao TT, Cheng N, Sun XD, Fu XP (2018) Towards underwater object recognition based on supervised learning. In: 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans, Kobe, pp 1–4. https://doi.org/10.1109/OCEANSKOBE.2018.8559050

[13] Ding XY, Wang YF, Zhang J, Fu XP (2017) Underwater image dehaze using scene depth estimation with adaptive color correction. In: OCEANS 2017-Aberdeen, Aberdeen, pp 1–5. https://doi.org/10.1109/OCEANSE.2017.8084665

[14] Duan YE, Li DL, Li ZB, Fu ZT (2015) Review on visual attributes measurement research of aquatic animals based on computer vision. Trans Chin Soc Agric Eng 31(15):1–11. https://doi.org/10.11975/j.issn.1002-6819.2015.15.001 (in Chinese with English abstract)

[15] Fan BJ, Chen W, Cong Y, Tian JD (2020) Dual refinement underwater object detection network. In: 16th European Conference on Computer Vision, Glasgow, pp 275–291. https://doi.org/10.1007/978-3-030-58565-5_17

[16] Fatan M, Daliri MR, Shahri AM. Underwater cable detection in the images using edge classification based on texture information. Measurement, 2016, 91: 309-317,

[17] Ge HL, Dai YW, Zhu ZY, Liu RB. A deep learning model applied to optical image target detection and recognition for the identification of underwater biostructures. Machines, 2022, 10(9): 809,

[18] Ge HL, Dai YW, Zhu ZY, Zang X. Single-stage underwater target detection based on feature anchor frame double optimization network. Sensors, 2022, 22(20): 7875, pmcid: 9608072

[19] Ghafoor H, Noh Y. An overview of next-generation underwater target detection and tracking: an integrated underwater architecture. IEEE Access, 2019, 7: 98841-98853,

[20] Gillis DB. An underwater target detection framework for hyperspectral imagery. IEEE J Sel Top Appl Earth Observ Remote Sens, 2020, 13: 1798-1810,

[21] Girshick R (2015) Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision, Santiago, pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

[22] Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, pp 580–587. https://doi.org/10.1109/CVPR.2014.81

[23] Gordan M, Dancea O, Stoian I, Georgakis A, Tsatos O (2006) A new SVM-based architecture for object recognition in color underwater images with classification refinement by shape descriptors. In: 2006 IEEE International Conference on Automation, Quality and Testing, Robotics, Cluj-Napoca, pp 327–332. https://doi.org/10.1109/AQTR.2006.254654

[24] Han FL, Yao JZ, Zhu HT, Wang CH. Underwater image processing and object detection based on deep CNN method. J Sens, 2020, 2020: 6707328,

[25] Han KM, Choi HT (2011) Shape context based object recognition and tracking in structured underwater environment. In: 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, pp 617–620. https://doi.org/10.1109/IGARSS.2011.6049204

[26] Hong J, Fulton M, Sattar J (2020) Trashcan: a semantically-segmented dataset towards visual detection of marine debris. Preprint at arXiv:2007.08097

[27] Hu XL, Liu Y, Zhao ZX, Liu JT, Yang XT, Sun CH, et al.. Real-time detection of uneaten feed pellets in underwater images for aquaculture using an improved YOLO-V4 network. Comput Electron Agric, 2021, 185: 106135,

[28] Jian MW, Lam KM, Dong JY, Shen LL. Visual-patch-attention-aware saliency detection. IEEE Trans Cybern, 2014, 45(8): 1575-1586,

[29] Jian MW, Liu XY, Luo HJ, Lu XW, Yu H, Dong JY. Underwater image processing and analysis: a review. Signal Proc: Image Commun, 2021, 91: 116088,

[30] Jian MW, Qi Q, Dong JY, Yin YL, Lam KM. Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection. J vis Commun Image Represent, 2018, 53: 31-41,

[31] Jian MW, Qi Q, Yu H, Dong JY, Cui CR, Nie XS, et al.. The extended marine underwater environment database and baseline evaluations. Appl Soft Comput, 2019, 80: 425-437,

[32] Jian MW, Zhang WY, Yu H, Cui CR, Nie XS, Zhang HX, et al.. Saliency detection based on directional patches extraction and principal local color contrast. J vis Commun Image Represent, 2018, 57: 1-11,

[33] Komari Alaie H, Farsi H. Passive sonar target detection using statistical classifier and adaptive threshold. Appl Sci, 2018, 8(1): 61,

[34] Lau PY, Lai SC (2021) Localizing fish in highly turbid underwater images. In: International Workshop on Advanced Imaging Technology (IWAIT), pp 294–299. https://doi.org/10.1117/12.2590995

[35] Lei F, Tang FF, Li SH. Underwater target detection algorithm based on improved YOLOv5. J Mar Sci Eng, 2022, 10(3): 310,

[36] Li CY, Guo JC, Cong RM, Pang YW, Wang B. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans Image Proc, 2016, 25(12): 5664-5677,

[37] Li X, Hao J, Shang M, Yang Z (2016b) Saliency segmentation and foreground extraction of underwater image based on localization. In: OCEANS 2016-Shanghai, Shanghai, pp 1–4. https://doi.org/10.1109/OCEANSAP.2016.7485498

[38] Lin WH, Zhong JX, Liu S, Li T, Li G (2020) ROIMIX: proposal-fusion among multiple images for underwater object detection. In: 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, pp 2588–2592. https://doi.org/10.1109/ICASSP40776.2020.9053829

[39] Liu CW, Li HJ, Wang SC, Zhu M, Wang D, Fan X et al (2021a) A dataset and benchmark of underwater object detection for robot picking. In: 2021 IEEE International Conference on Multimedia & Expo Workshops, Shenzhen, pp 1–6. https://doi.org/10.1109/ICMEW53276.2021.9455997

[40] Liu CW, Wang ZH, Wang SJ, Tang T, Tao YL, Yang CF, et al.. A new dataset, Poisson GAN and AquaNet for underwater object grabbing. IEEE Trans Circuits Syst Video Technol, 2021, 32(5): 2831-2844,

[41] Liu RS, Fan X, Zhu M, Hou MJ, Luo ZX. Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol, 2020, 30(12): 4861-4875,

[42] Liu RS, Jiang ZY, Yang SZ, Fan X. Twin adversarial contrastive learning for underwater image enhancement and beyond. IEEE Trans Image Proc, 2022, 31: 4922-4936,

[43] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY et al (2016) SSD: single shot multibox detector. In: 14th European Conference on Computer Vision (ECCV), Amsterdam, pp 21–37. https://doi.org/10.1007/978-3-319-46448-0_2

[44] Mou L, Zhang XW, Zhang JJ, Shen XH, Xu XL (2017) Saliency detection of underwater target based on spatial probability. In: 2017 International Conference on Computer Systems, Electronics and Control, Dalian, pp 630–632. https://doi.org/10.1109/ICCSEC.2017.8446733

[45] Mukherjee K, Gupta S, Ray A, Phoha S. Symbolic analysis of sonar data for underwater target detection. IEEE J Ocean Eng, 2011, 36(2): 219-230,

[46] Nagaraja S, Prabhakar CJ, Kumar PUP (2015) Extraction of texture based features of underwater images using RLBP descriptor. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications, Bhubaneswar, pp 263–272. https://doi.org/10.1007/978-3-319-12012-6_29

[47] Pedersen M, Bruslund Haurum J, Gade R, Moeslund TB (2019) Detection of marine animals in a new underwater dataset with varying visibility. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, pp 18–26

[48] Peng XH, Liang ZX, Zhang J, Chen RF. Review of underwater image preprocessing based on deep learning. Comput Eng Appl, 2021, 57(13): 43-54 (in Chinese with English abstract)

[49] Rashwan A, Kalra A, Poupart P (2019) Matrix Nets: a new deep architecture for object detection. In: 2019 IEEE/CVF International Conference on Computer Vision Workshops, Seoul, pp 2025–2028. https://doi.org/10.1109/ICCVW.2019.00252

[50] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2015, 39(6): 1137-1149,

[51] Shi XT, Huang H, Wang B, Pang S, Qin HD (2019) Underwater cage boundary detection based on GLCM features by using SVM classifier. In: 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Hong Kong, pp 1169–1174. https://doi.org/10.1109/AIM.2019.8868517

[52] Singh P, Deepak BBVL, Sethi T, Murthy MDP (2015) Real-time object detection and tracking using color feature and motion. In: 2015 International Conference on Communications and Signal Processing, Melmaruvathur, pp 1236–1241. https://doi.org/10.1109/ICCSP.2015.7322705

[53] Song DL, Sun WC, Ji ZH, Hou GJ, Li XF, Liu L (2014) Color model selection for underwater object recognition. In: 2014 International Conference on Information Science, Electronics and Electrical Engineering, Sapporo, pp 1339–1342. https://doi.org/10.1109/InfoSEEE.2014.6947890

[54] Srividhya K, Ramya MM. Accurate object recognition in the underwater images using learning algorithms and texture features. Multimed Tools Appl, 2017, 76: 25679-25695,

[55] Sun X, Shi JY, Liu LP, Dong JY, Plant C, Wang XH, et al.. Transferring deep knowledge for object recognition in Low-quality underwater videos. Neurocomputing, 2018, 275: 897-908,

[56] Susanto T, Mardiyanto R, Purwanto D (2018) Development of underwater object detection method base on color feature. In: 2018 International Conference on Computer Engineering, Network and Intelligent Multimedia, Surabaya, pp 254–259. https://doi.org/10.1109/CENIM.2018.8711290

[57] Taud H, Mas JF (2018) Multilayer perceptron (MLP). In: Geomatic approaches for modeling land change scenarios. Springer, Cham, pp 451–455. https://doi.org/10.1007/978-3-319-60801-3_27

[58] Tucker JD, Azimi-Sadjadi MR. Coherence-based underwater target detection from multiple disparate sonar platforms. IEEE J Ocean Eng, 2011, 36(1): 37-51,

[59] Wang HB, Zhang Q, Wang X, Chen Z. Object detection based on regional saliency and underwater optical priors. Chin J Sci Instrum, 2014, 35(2): 387-397, in Chinese with English abstract)

[60] Wei XY, Yu L, Tian SW, Feng PC, Ning X. Underwater target detection with an attention mechanism and improved scale. Multimed Tools Appl, 2021, 80: 33747-33761,

[61] Wu Y, Cai YB, Tang RH. Research on the underwater optical imaging processing and identification. Ship Electron Eng, 2019, 39(5): 93-96 (in Chinese with English abstract)

[62] Yang HH, Xu GH, Yi SZ, Li YQ (2019) A new cooperative deep learning method for underwater acoustic target recognition. In: OCEANS 2019-Marseille, Marseille, pp 1–4. https://doi.org/10.1109/OCEANSE.2019.8867490

[63] Yu H. Research progresson object detection and tracking techniques utilization in aquaculture: a review. J Dalian Ocean Univ, 2020, 35(6): 793-804 (in Chinese with English abstract)

[64] Yu XL, Qu YY, Hong M (2019) Underwater-GAN: underwater image restoration via conditional generative adversarial network. In: 24th International Conference on Pattern Recognition (ICPR), Beijing, pp 66–75. https://doi.org/10.1007/978-3-030-05792-3_7

[65] Yuan X, Guo LX, Luo CT, Zhou XT, Yu CL. A survey of target detection and recognition methods in underwater turbid areas. Appl Sci, 2022, 12(10): 4898,

[66] Zeng LC, Sun B, Zhu DQ. Underwater target detection based on Faster R-CNN and adversarial occlusion network. Eng Appl Artif Intell, 2021, 100: 104190,

[67] Zhang MH, Xu SB, Song W, He Q, Wei QM. Lightweight underwater object detection based on YOLO v4 and multi-scale attentional feature fusion. Remote Sens, 2021, 13(22): 4706,

[68] Zhou XY, Yang KD, Duan R. Deep learning based on striation images for underwater and surface target classification. IEEE Signal Proc Lett, 2019, 26(9): 1378-1382,

[69] Zhu YF, Chang L, Dai JL, Zheng HY, Zheng B (2016) Automatic object detection and segmentation from underwater images via saliency-based region merging. In: OCEANS 2016-Shanghai, Shanghai, pp 1–4. https://doi.org/10.1109/OCEANSAP.2016.7485598

[70] Zurowietz M, Nattkemper TW. Unsupervised knowledge transfer for object detection in marine environmental monitoring and exploration. IEEE Access, 2020, 8: 143558-143568,

Funding

Key Development Program for Basic Research of Shandong Province((ZR2020ZD44))