Perceptual loss function for generating high-resolution climate data

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

When planning the development of future energy resources, electrical infrastructure, transportation networks, agriculture, and many other societally significant systems, policy makers require accurate and high-resolution data reflecting different climate scenarios. There is widely documented evidence that perceptual loss can be used to generate perceptually realistic results when mapping low-resolution inputs to high-resolution outputs, but its application is limited to images at present. In this paper, we study the perceptual loss when increasing the resolution of raw precipitation data by ×4 and ×8 under training modes of CNN and GAN. We examine the difference in the perceptual loss calculated by using different layers of feature maps and demonstrate how low-and mid-level feature maps can yield comparable results to pixel-wise loss. In particular, from both qualitative and quantitative points of view, Conv2_1 and Conv3_1 are the best compromises between obtaining detailed information and maintaining the overall error in our case. We propose a new approach to benefit from perceptual loss while considering the characteristics of climate data. We show that in comparison to calculating perceptual loss directly for the entire sample, our proposed approach can obtain detailed information of extreme events regions while reducing error.

Keywords

climate downscaling / perceptual loss / deep learning / GAN

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] F. K. Muthoni, V. O. Odongo, J. Ochieng, E. M. Mugalavai, S. K. Mourice, I. Hoeschezeledon, et al., Longterm spatialtemporal trends and variability of rainfall over Eastern and Southern Africa. Theor. Appl. Climatol., 137 (2019), 1869-1882. https://doi.org/10.1007/s00704-018-2712-1

[2] C. C. Wei, Simulation of operational typhoon rainfall nowcasting using radar reflectivity combined with meteorological data, J. Geophys. Res.Atmos., 119 (2014), 6578-6595. https://doi.org/10.1002/2014JD021488

[3] C. P. Castillo, F. B. e Silva, C. Lavalle, An assessment of the regional potential for solar power generation in EU28, Energy Policy, 88 (2016), 86-99. https://doi.org/10.1016/j.enpol.2015.10.004

[4] A. Voldoire, E. SanchezGomez, D. Salas y Mélia, B. Decharme, C. Cassou, S. Sénési, et al., The CNRMCNRMCM5.1 global climate model: Description and basic evaluation, Clim. Dyn., 40 (2013), 2091-2121. https://doi.org/10.1007/s00382-011-1259-y

[5] N. Hofstra, M. Haylock, M. New, P. Jones, C. Frei, Comparison of six methods for the interpolation of daily, European climate data, J. Geophys. Res. Atmos., 113 (2008). https://doi.org/10.1029/2008JD010100

[6] C. Dong, C. C. Loy, K. He, X. Tang, Image SuperResolution Using Deep Convolutional Networks, IEEE T. Pattern Anal. Mach. Intell., 38 (2015), 295-307. https://doi.org/10.1109/TPAMI.2015.2439281

[7] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, et al., Photorealistic single image superresolution using a generative adversarial network, Proc. 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, 4681-4690. https://doi.org/10.1109/CVPR.2017.19

[8] K. Stengel, A. Glaws, D. Hettinger, R. N. King, Adversarial superresolution of climatological wind and solar data, Proc. Natl. Acad. Sci. U. S. A., 117 (2020), 16805-16815. https://doi.org/10.1073/pnas.1918964117

[9] T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, A. R. Ganguly, Generating high resolution climate change projections through single image superresolution: An abridged version, IJCAI Int. Jt. Conf. Artif. Intell., (2018) 5389-5393. https://doi.org/10.24963/ijcai.2018/759

[10] E. R. Rodrigues, I. Oliveira, R. Cunha, M. Netto, DeepDownscale: A deep learning strategy for highresolution weather forecast, Proc. IEEE 14th Int. Conf. eScience, eScience, (2018), 415-422. https://doi.org/10.1109/eScience.2018.00130

[11] Y. Jo, S. Yang, S. J. Kim, Investigating loss functions for extreme superresolution, IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., (2020), 424-425.

[12] J. Johnson, A. Alahi, L. FeiFei, Perceptual losses for realtime style transfer and superresolution, Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 9906 (2016), 694-711. https://doi.org/10.1007/978-3-319-46475-6_43

[13] M. S. Sajjadi, B. Scholkopf, M. Hirsch, EnhanceNet: Single Image SuperResolution Through Automated Texture Synthesis, Proc. IEEE Int. Conf. Comput. Vis., (2017), 4491-4500. https://doi.org/10.1109/ICCV.2017.481

[14] K. Simonyan, A. Zisserman, Very deep convolutional networks for largescale image recognition, 3rd Int. Conf. Learn. Represent. ICLR 2015 Conf. Track Proc., (2015), 1-14.

[15] A. Mahendran, A. Vedaldi, Visualizing Deep Convolutional Neural Networks Using Natural Preimages, Int. J. Comput. Vis., 120 (2016), 233-255. https://doi.org/10.1007/s11263-016-0911-8

[16] W. Yu, K. Yang, Y. Bai, H. Yao, Y. Rui, Visualizing and Comparing Convolutional Neural Networks, 2014.

[17] Y. Ma, K. A. Lee, V. Hautamaki, H. Li, PLEESR: Perceptual Loss Based EndtoEnd Robust Speaker Representation Extraction, 2021 IEEE Autom. Speech Recognit. Underst. Work. ASRU, (2021), 106-113.

[18] E. Shelhamer, J. Long, T. Darrell, Fully Convolutional Networks for Semantic Segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015), 3431-3440. https://doi.org/10.1109/CVPR.2015.7298965

[19] Y. Liu, H. Chen, Y. Chen, W. Yin, C. Shen, Generic Perceptual Loss for Modeling Structured Output Dependencies, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., (2021), 5420-5428. https://doi.org/10.1109/CVPR46437.2021.00538

[20] R. Huth, Statistical downscaling of daily temperature in central Europe, J. Clim., 15 (2002), 1731-1742. https://doi.org/10.1175/1520-0442(2002)015<1731:SDODTI>2.0.CO;2

[21] S. T. Chen, P. S. Yu, Y. H. Tang, Statistical downscaling of daily precipitation using support vector machines and multivariate analysis, J. Hydrol., 385 (2010), 13-22. https://doi.org/10.1016/j.jhydrol.2010.01.021

[22] C. Hutengs, M. Vohland, Downscaling land surface temperatures at regional scales with random forest regression, Remote Sens. Environ., 178 (2016), 127-141. https://doi.org/10.1016/j.rse.2016.03.006

[23] T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, A. R. Ganguly, DeepSD: Generating high resolution climate change projections through single image superresolution, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., (2017), 1663-1672. https://doi.org/10.1145/3097983.3098004

[24] J. Cheng, Q. Kuang, C. Shen, J. Liu, X. Tan, W. Liu, ResLap: Generating HighResolution Climate Prediction through Image SuperResolution, IEEE Access, 8 (2020), 39623-39634. https://doi.org/10.1109/ACCESS.2020.2974785

[25] C. Besombes, O. Pannekoucke, C. Lapeyre, B. Sanderson, O. Thual, Producing realistic climate data with generative adversarial networks, Nonlinear Proc. Geoph., 28 (2021), 347-370. https://doi.org/10.5194/npg-28-347-2021

[26] J. Bruna, P. Sprechmann, Y. LeCun, Superresolution with deep convolutional sufficient statistics, 4th Int. Conf. Learn. Represent. ICLR 2016 Conf. Track Proc., (2016), 1-17.

[27] X. Xu, M. Xie, P. Miao, W. Qu, W. Xiao, H. Zhang, et al., PerceptualAware Sketch Simplification Based on Integrated VGG Layers, IEEE T. Vis. Comput. Gr., 27 (2019), 178-189. https://doi.org/10.1109/TVCG.2019.2930512

[28] A. Mahendran, A. Vedaldi, Understanding deep image representations by inverting them, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., (2015), 5188-5196. https://doi.org/10.1109/CVPR.2015.7299155

[29] Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, et al., LowDose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss, IEEE T. Med. Imaging, 37 (2018), 1348-1357. https://doi.org/10.1109/TMI.2018.2827462

[30] P. Sangkloy, J. Lu, C. Fang, F. Yu, J. Hays, Scribbler: Controlling deep image synthesis with sketch and color, Proc. 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, (2017), 6836-6845. https://doi.org/10.1109/CVPR.2017.723

[31] M. S. Rad, B. Bozorgtabar, U. V. Marti, M. Basler, H. K. Ekenel, J. P. Thiran, SROBB: Targeted perceptual loss for single image superresolution, Proc. IEEE Int. Conf. Comput. Vis., (2019), 2710-2719.

[32] Y. Zhang, W. Li, W. Gong, Z. Wang, J. Sun, An improved boundaryaware perceptual loss for building extraction from VHR images, Remote Sens., 12 (2020), 1195. https://doi.org/10.3390/rs12071195

[33] J. Chi, J. Bae, Y. J. Kwon, Twostream convolutional longand shortterm memory model using perceptual loss for sequencetosequence arctic sea ice prediction, Remote Sens., 13 (2021), 3413. https://doi.org/10.3390/rs13173413

[34] G. SkofronickJackson, W. A. Petersen, W. Berg, C. Kidd, E. F. Stocker, D. B. Kirschbaum, et al., The global precipitation measurement (GPM) mission for science and Society, B. Am. Meteorol. Soc., 98 (2017), 1679-1695.

[35] V. Levizzanni, P. Bauer, F. J. Turk, Measuring Precipitation from Space: EURAINSAT and the Future, vol. 28, 2007. https://doi.org/10.1007/978-1-4020-5835-6

[36] J. P. Dunne, L. W. Horowitz, A. J. Adcroft, P. Ginoux, I. M. Held, J. G. John, et al., The GFDL Earth System Model Version 4.1 (GFDLESM 4.1): Overall Coupled Model Description and Simulation Characteristics, J. Adv. Model. Earth Syst., 12 (2020), e2019MS002015. https://doi.org/10.1029/2019MS002015