Analysis and comparison for image colorization with machine learning based on PyTorch and ChromaGAN

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

Colorization of grayscale images has found widespread applications across artistic, historical, scientific, medical, and industrial domains. Traditional manual colorization methods, however, are labor-intensive, time-consuming, and prone to subjective interpretation. In recent years, various deep learning (DL) approaches have been explored to automate the colorization process. Despite this progress, the efficacy and accuracy of these DL methods remain largely unexamined. One of the primary challenges is the accurate handling of color boundaries, which significantly affects the realism and visual clarity of the colorized output. In this study, we investigated the impact of different DL algorithms and training epochs on the quality of colorized images, comparing them to their corresponding ground truths. To explore these effects, a semantic-guided adversarial generative framework was employed, and five commonly used DL algorithms were compared.

Keywords

image colorization / deep learning / ChromaGAN

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] S. Huang, X. Jin, Q. Jiang, L. Liu, Deep learning for image colorization: Current and future prospects, Eng. Appl. Artif. Intell., 114 (2022), 105006. https://doi.org/10.1016/j.engappai.2022.105006

[2] P. Vitoria, L. Raad, C. Ballester, Chromagan: Adversarial picture colorization with semantic class distribution, in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), (2020), 2434-2443. https://doi.org/10.1109/WACV45572.2020.9093389

[3] A. Levin, D. Lischinski, Y. Weiss, Colorization using optimization, ACM Trans. Graphics, 23 (2004), 689-694. https://doi.org/10.1145/1015706.1015780

[4] Y. C. Huang, Y. S. Tung, J. C. Chen, S. W. Wang, J. L. Wu, An adaptive edge detection based colorization algorithm and its applications, in Proceedings of the 13th annual ACM International Conference on Multimedia, (2005), 351-354. https://doi.org/10.1145/1101149.1101223

[5] L. Yatziv, G. Sapiro, Fast image and video colorization using chrominance blending, IEEE Trans. Image Process., 15 (2006), 1120-1129. https://doi.org/10.1109/TIP.2005.864231

[6] Q. Luan, F. Wen, D. Cohen-Or, L. Liang, Q. Xu, Y. Shum, Natural image colorization, in Proceedings of the 18th Eurographics Conference on Rendering Techniques, (2007), 309-320.

[7] T. Welsh, M. Ashikhmin, K. Mueller, Transferring color to greyscale images, ACM Trans. Graphics, 21 (2002), 277-280. https://doi.org/10.1145/566654.566576

[8] R. Ironi, D. Cohen-Or, and D. Lischinski . Colorization by example, in Rendering Techniques, (2005), 201-210.

[9] Y. W. Tai, J. Jia, K. Tang, Local color transfer via probabilistic segmentation by expectation-maximization, in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 1 (2005), 747-754. https://doi.org/10.1109/CVPR.2005.215

[10] G. Charpiat, M. Hofmann, B. Scholkopf, Automatic image colorization via multimodal predictions, in Computer Vision - ECCV 2008, (2008), 126-139. https://doi.org/10.1007/978-3-540-88690-7_10

[11] A. Y. Chia, S. Zhuo, R. K. Gupta, Y. W. Tai, S. Y. Cho, P. Tan, et al., Semantic colorization with internet images, ACM Trans. Graphics, 30 (2011), 1-8. https://doi.org/10.1145/2070781.2024190

[12] R. K. Gupta, A. Y. Chia, D. Rajan, E. S. Ng, Y. Huang, Image colorization using similar images, in Proceedings of the 20th ACM International Conference on Multimedia, (2012), 369-378. https://doi.org/10.1145/2393347.2393402

[13] Z. Cheng, Q. Yang, B. Sheng, Deep colorization, in Proceedings of the IEEE International Conference on Computer Vision, (2015), 415-423. https://doi.org/10.1109/ICCV.2015.55

[14] S. Iizuka, E. Simo-Serra, H. Ishikawa, Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification, ACM Trans. Graphics, 35 (2016), 110. https://doi.org/10.1145/2897824.2925974

[15] R. Zhang, P. Isola, A. A. Efros, Colorful image colorization, in Computer Vision - ECCV 2016, (2016), 649-666.

[16] M. H. Noaman, H. Khaled, H. M. Faheem, Image colorization: A survey of methodologies and techniques, in Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2021, (2021), 115-130. https://doi.org/10.1007/978-3-030-89701-7_11

[17] S. Anwar, M. Tahir, C. Li, A. Mian, F. S. Khan, A. W. Muzaffar, Image colorization: A survey and dataset, Inf. Fusion, 114 (2025), 102720. https://doi.org/10.1016/j.inffus.2024.102720

[18] R. Zhang, J. Y. Zhu, P. Isola, X. Geng, S. Lin, T. Yu, et al., Real-time user-guided image colorization with learned deep priors, ACM Trans. Graphics, 36 (2017), 119. https://doi.org/10.1145/3072959.3073703

[19] M. He, D. Chen, J. Liao, P. V. Sander, L. Yuan, Deep exemplar-based colorization, ACM Trans. Graphics, 37 (2018), 47. https://doi.org/10.1145/3197517.3201365

[20] P. Isola, J. Y. Zhu, T. Zhou, A. A. Efros, Image-to-image translation with conditional adversarial networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 5967-5976.

[21] Y. Cao, Z. Zhou, W. Zhang, Y. Yu, Unsupervised diverse colorization via generative adversarial networks, in Machine Learning and Knowledge Discovery in Databases, (2017), 151-166.

[22] S. Yoo, H. Bahng, S. Chung, J. Lee, J. Chang, J. Choo, Coloring with limited data: Few-shot colorization via memory augmented networks, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 11283-11292.

[23] M. Arjovsky, S. Chintala, L. Bottou, Wasserstein generative adversarial networks, in Proceedings of the 34th International Conference on Machine Learning, (2017), 214-223.

[24] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, A. C. Courville, Improved training of Wasserstein GANs, in Advances in Neural Information Processing Systems, 30 (2017).

[25] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv:1409.1556.

[26] A. Odena, V. Dumoulin, C. Olah, Deconvolution and checkerboard artifacts, Distill, 2016.

[27] M. D. Zeiler, G. W. Taylor, R. Fergus, Adaptive deconvolutional networks for mid and high level feature learning, in 2011 International Conference on Computer Vision, (2011), 2018-2025. https://doi.org/10.1109/ICCV.2011.6126474

[28] M. Zhang, M. Li, J. Yu, L. Chen, Aesthetic photo collage with deep reinforcement learning, IEEE Trans. Multimedia, 25 (2023), 4653-4664.

[29] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, The unreasonable effectiveness of deep features as a perceptual metric, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 586-595.