BROWSE
Detail
Conference Paper
From 2D Portraits to 3D Realities: Advancing GAN Inversion for Enhanced Image Synthesis
- Authors
- Wonseok Oh, Youngjoo Jo
- Issue Date
- 2024-06
- Citation
- Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2024, pp.737-746
- Language
- English
- Type
- Conference Paper
- Abstract
- Image synthesis using StyleGAN has shown remarkable results in 2D portrait image generation. The works of the GAN inversion to manipulate the real image using StyleGAN latent space also show remarkable achievements. 2D GAN inversion has successfully manipulated global attributes such as facial expressions and gender. However, preserving the hairstyle and identity was difficult according to the pose change. We introduce the 3D GAN inversion encoder to make a high-resolution 3D image based on the Geometry Aware 3D Generative Adversarial Network, known as EG3D, which allows explicit control over the pose of the real image subject with multi-view consistency. Our network projects the single 2D portrait images to novel latent space for 3D GAN inversion for the tri-plane of EG3D. We also present multi-view cycle loss, which aims to increase multi-view consistency. By leveraging the new latent space and loss for 3D GAN inversion, our network can successfully convert 2D portrait images into 3D fast.
- KSP Keywords
- 3D images, Enhanced image, Facial expression, Global attributes, High resolution, Latent space, Multi-view, Portrait image, explicit control, generative adversarial network, image generation