상세정보
학술지
WeatherGAN: Unsupervised multi-weather image-to-image translation via single content-preserving UResNet generator
- 저자
- 황선희, 전석규, 마유승, 변혜란
- 발행일
- 202211
- 출처
- Multimedia Tools and Applications, v.81 no.28, pp.40269-40288
- ISSN
- 1380-7501
- 출판사
- Springer
- 협약과제
- 22HS2300, 인공지능 시스템을 위한 뉴로모픽 컴퓨팅 SW 플랫폼 기술 개발, 김태호
- 초록
- In this paper, we propose an unsupervised and unified multi-domain Image-to-Image translation model for an image weather domain translation. Most existing multi-domain Image-to-Image translation methods are capable of translating fine details such as facial attributes. However, the image translation model between multiple weather domains, e.g., sunny-to-snowy, or sunny-to-rainy, have to consider the large domain gap. To address the challenging problem, in this paper, we propose WeatherGAN based on a proposed UResNet generator. Our model consists of the UResNet generator, a PatchGAN discriminator, and a VGG perceptual encoder. UResNet is a combined model of U-Net and ResNet to address the ability of each model, that preserve input context information and generate realistic images. The PatchGAN discriminator encourages the generator to produce realistic images of the target domain by criticizing patch-wise details. We also leverage VGG perceptual encoder as a loss network, which guides the generator to minimize the perceptual distance between an input image and generated images to enhance the quality of outputs. Through the extensive experiments on Alps, YouTube driving (our benchmark dataset), and BDD datasets, we demonstrate that WeatherGAN produces more satisfactory results of the target domain compared to the baselines. Besides, we also conduct a data augmentation task to show the usability of our generated images by WeatherGAN, and it shows the overall object detection performance of YOLO v3 is improved in our results on BDD dataset.
- KSP 제안 키워드
- Benchmark datasets, Context Information, Data Augmentation, Facial Attributes, Fine details, Large domain, Loss network, Multi-Domain, Object detection, Patch-wise, Perceptual distance