Water2LandNet: generative adversarial networks for UAV image dewatering

Abstract

Restoring coastal images from high tide to low tide conditions is crucial for applications such as environmental monitoring, coastal planning, and disaster management. Image dewatering, the process of transforming high-tide images into their corresponding low-water-level states to reveal submerged objects, remains a challenging task. Unmanned aerial vehicle (UAV)-based image dewatering presents several unique difficulties. First, there is a scarcity of suitable pre-existing datasets for supervised learning and model training, necessitating extensive data collection to construct an appropriate dataset. Additionally, aligning and matching image pairs is complex. Due to the operational characteristics of UAVs, precisely aligning high-altitude images captured at the same geographic location is difficult, which negatively impacts model training and evaluation. To address these challenges, this study proposes Water2LandNet, an unsupervised UAV image dewatering model based on generative adversarial networks (GANs), which enables training with unpaired images. This approach effectively resolves the data-pairing problem present in our dewatering dataset, OUCD (OUC_UAV_DEWATER). Extensive experiments on the OUCD dataset show that the proposed model effectively removes water from UAV images, as evidenced by qualitative and quantitative evaluations. Furthermore, compared with traditional supervised methods, Water2LandNet demonstrates greater robustness to texture inconsistencies and illumination variations arising from dynamic marine environments. Experimental results confirm that the model can reconstruct visually realistic low-tide scenes even when trained on limited unpaired data, offering a practical solution for large-scale coastal mapping and post-disaster reconstruction.

Keywords

Low-tide image enhancement / GAN / Underwater images / Image signal processing

References

[1] Abdal R, Qin YP, Wonka P (2019) Image2StyleGAN: how to embed images into the StyleGAN latent space. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, pp 4431–4440. https://doi.org/10.1109/iccv.2019.00453

[2] Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning. PMLR, pp 214–223. https://proceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf

[3] Brock A, Donahue J, Simonyan K (2019) Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations. ICLR, pp 1–35. https://openreview.net/forum?id=B1xsqj09Fm

[4] Cai BL, Xu XM, Jia K, Qing CM, Tao DC. DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Proc, 2016, 25(11): 5187-5198

[5] Dong H, Pan JS, Xiang L, Hu Z, Zhang XY, Wang F et al (2020) Multi-scale boosted dehazing network with dense feature fusion. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2154–2164. https://doi.org/10.1109/CVPR42600.2020.00223

[6] Fattal R. Dehazing using color-lines. ACM Trans Graph, 2014, 34(1): 1-14

[7] Gal R, Hochberg DC, Bermano A, Cohen-Or D. SWAGAN: a style-based wavelet-driven generative model. ACM Trans Graph, 2021, 40(4): 1-11

[8] Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S et al (2014) Generative adversarial nets. In: Proceedings of the 28th Conference on Neural Information Processing Systems (NIPS). Curran Associates Inc., pp 2672–2680. https://dl.acm.org/doi/10.5555/2969033.2969125

[9] Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of wasserstein GANs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., pp 5769–5779. https://dl.acm.org/doi/10.5555/3295222.3295327

[10] Han S, Srivastava A, Hurwitz C, Sattigeri P, Cox DD (2020) not-so-BigGAN: generating high-fidelity images on a small compute budget. Preprint at arXiv:2009.04433

[11] He KM, Sun JA, Tang XO. Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell, 2011, 33(12): 2341-2353

[12] Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., pp 6626–6640. https://dl.acm.org/doi/10.5555/3295222.3295408

[13] Huang X, Liu MY, Belongie S, Kautz J (2018) Multimodal unsupervised image-to-image translation. In: Ferrari V et al (eds) Computer Vision–ECCV 2018. Lecture notes in computer science, vol 11207. Springer, Cham, pp 179–196. https://doi.org/10.1007/978-3-030-01219-9_11

[14] Karras T, Aittala M, Hellsten J, Laine S, Lehtinen J, Aila T (2020a) Training generative adversarial networks with limited data. In: Proceedings of the 34th Conference on Neural Information Processing Systems. Curran Associates Inc., pp 12104–12114. https://proceedings.neurips.cc/paper_files/paper/2020/file/8d30aa96e72440759f74bd2306c1fa3d-Paper.pdf

[15] Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020b) Analyzing and improving the image quality of StyleGAN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 8107–8116. https://doi.org/10.1109/cvpr42600.2020.00813

[16] Kingma DP, Welling M (2013) Auto-encoding variational bayes. Preprint at arXiv:1312.6114

[17] Li BY, Gou YB, Gu SH, Liu JZ, Zhou JT, Peng X. You only look yourself: unsupervised and untrained single image dehazing neural network. Int J Comput Vis, 2021, 129(5): 1754-1767

[18] Lim JH, Ye JC (2017) Geometric GAN. Preprint at arXiv:1705.02894

[19] Liu BC, Zhu YZ, Song KP, Elgammal A (2021) Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis. Preprint at arXiv:2101.04775

[20] Liu XH, Ma YR, Shi ZH, Chen J (2019) GridDehazeNet: attention-based multi-scale network for image dehazing. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, pp 7313–7322. https://doi.org/10.1109/ICCV.2019.00741

[21] Menon S, Damian A, Hu SJ, Ravi N, Rudin C (2020) PULSE: self-supervised photo upsampling via latent space exploration of generative models. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2434–2442. https://doi.org/10.1109/CVPR42600.2020.00251

[22] Mescheder L, Geiger A, Nowozin S (2018) Which training methods for GANs do actually converge? In: Proceedings of the 35th International Conference on Machine Learning. PMLR, pp 3481–3490. https://proceedings.mlr.press/v80/mescheder18a.html

[23] Miyato T, Kataoka T, Koyama M, Yoshida Y (2018) Spectral normalization for generative adversarial networks. Preprint at arXiv:1802.05957

[24] Park T, Zhu JY, Wang O, Lu JW, Shechtman E, Efros AA et al (2020) Swapping autoencoder for deep image manipulation. Preprint at arXiv:2007.00653

[25] Pidhorskyi S, Adjeroh DA, Doretto G (2020) Adversarial latent autoencoders. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 14092–14101. https://doi.org/10.1109/CVPR42600.2020.01411

[26] Qin X, Wang ZL, Bai YC, Xie XD, Jia HZ (2020) FFA-Net: feature fusion attention network for single image dehazing. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. AAAI, pp 11908–11915. https://doi.org/10.1609/aaai.v34i07.6865

[27] Ren WQ, Liu S, Zhang H, Pan JS, Cao XC, Yang MH (2016) Single image dehazing via multi-scale convolutional neural networks. In: Leibe B et al (eds) Computer Vision–ECCV 2016. Lecture notes in computer science, vol 9906. Springer, Cham, pp 154–169. https://doi.org/10.1007/978-3-319-46475-6_10

[28] Sajjadi MSM, Bachem O, Lucic M, Bousquet O, Gelly S (2018) Assessing generative models via precision and recall. Preprint at arXiv:1806.00035

[29] Shao YJ, Li LRH, Ren WQ, Gao CX, Sang N (2020) Domain adaptation for image dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2805–2814. https://doi.org/10.1109/CVPR42600.2020.00288

[30] Thakur A, Satish M. Generative adversarial networks. Int J Res Appl Sci Eng Technol, 2021, 9(8): 2307-2325

[31] Wang SK, Wang XD, Qu L, Yao FQ, Liu YY, Li CH, et al.. Semantic segmentation benchmark dataset for coastal ecosystem monitoring based on unmanned aerial vehicle (UAV). J Image Graph, 2024, 29(8): 2162-2174

[32] Wei YY, Zhang Z, Wang Y, Xu ML, Yang Y, Yan SC, et al.. DerainCycleGAN: rain attentive CycleGAN for single image deraining and rainmaking. IEEE Trans Image Process, 2021, 30: 4788-4801

[33] Yang Y, Wang CY, Liu RS, Zhang L, Guo XJ, Tao DC (2022) Self-augmented unpaired image dehazing via density and depth decomposition. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2027–2036. https://doi.org/10.1109/CVPR52688.2022.00208

[34] Yi ZL, Zhang H, Tan P, Gong ML (2017) DualGAN: unsupervised dual learning for image-to-image translation. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 2849–2876. https://doi.org/10.1109/ICCV.2017.310

[35] Zhang H, Zhang ZZ, Odena A, Lee H (2019) Consistency regularization for generative adversarial networks. Preprint at arXiv:1910.12027

[36] Zhao M, Bao F, Li CX, Zhu J (2022) EGSDE: unpaired image-to-image translation via energy-guided stochastic differential equations. Preprint at arXiv:2207.06635

[37] Zhao YH, Wu RH, Dong H (2020) Unpaired image-to-image translation using adversarial consistency loss. In: Vedaldi A et al (eds) Computer Vision–ECCV 2020. Lecture notes in computer science, vol 12354. Springer, Cham, pp 800–815. https://doi.org/10.1007/978-3-030-58545-7_46

[38] Zheng WF, Li Q, Zhang GX, Wan PF, Wang ZY (2022) ITTR: unpaired image-to-image translation with transformers. Preprint at arXiv:2203.16015

[39] Zheng Y, Zhan JH, He SF, Dong JY, Du Y (2023) Curricular contrastive regularization for physics-aware single image dehazing. Preprint at arXiv:2303.14218

[40] Zhou S, Gordon ML, Krishna R, Narcomey A, Li FF, Bernstein MS (2019) HYPE: a benchmark for human eye perceptual evaluation of generative models. Preprint at arXiv:1904.01121

[41] Zhu QS, Mai JM, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process, 2015, 24(11): 3522-3533

[42] Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 2242–2251. https://doi.org/10.1109/iccv.2017.244

[43] Zhu HY, Peng X, Chandrasekhar V, Li LY, Lim JH (2018) DehazeGAN: when image dehazing meets differential programming. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI). IJCAI, pp 1234–1240. https://doi.org/10.24963/ijcai.2018/172

RIGHTS & PERMISSIONS

The Author(s)