Enhanced Panoramic Image Generation with GAN and CLIP Models

Abstract

Panoramic images, offering a 360-degree view, are essential in virtual reality (VR) and augmented reality (AR), enhancing realism with high-quality textures. However, acquiring complete and high-quality panoramic textures is challenging. This paper introduces a method using generative adversarial networks (GANs) and the contrastive language-image pretraining (CLIP) model to restore and control texture in panoramic images. The GAN model captures complex structures and maintains consistency, while CLIP enables fine-grained texture control via semantic text-image associations. GAN inversion optimizes latent codes for precise texture details. The resulting low dynamic range (LDR) images are converted to high dynamic range (HDR) using the Blender engine for seamless texture blending. Experimental results demonstrate the effectiveness and flexibility of this method in panoramic texture restoration and generation.

Keywords

panoramic images / environment texture / generative adversarial networks (GANs) / contrastive language-image pretraining(CLIP) model / blender engine / fine-grained control / texture generation