BROWSE
Detail
Conference Paper
Real-Time Photorealistic Style Transfer of Digital Humans for Immersive Virtual Reality
Cited 0 time in 
Share 
Share
- Issue Date
- 2025-10
- Citation
- International Symposium on Mixed and Augmented Reality (ISMAR) 2025, pp.793-803
- Language
- English
- Type
- Conference Paper
- Abstract
- We present a novel approach for real-time photorealistic style transfer of digital humans in virtual reality environments using a lightweight U-Net-based neural network architecture. Our method transforms rendered VR images into photorealistic images while maintaining temporal consistency. Unlike previous approaches that attempt to support arbitrary style transfer, we focus on predefined target styles, enabling significantly higher performance and visual fidelity in real-time applications. Our technique achieves high frame rates (104 FPS at 2K × 2K resolution) through optimization and 8-bit integer quantization with NVIDIA's TensorRT. By incorporating foveated rendering techniques that prioritize processing in the center of vision, we further achieve 72+ FPS at 2 × 2064 × 2208 (stereo resolution) when integrated into a full PC-powered VR pipeline. The temporal artifacts such as flickering are eliminated through our direct image-to-image regression training, without additional temporal constraints. We evaluate two training methodologies: application-specific using sampled VR renderings, and generalized using diverse photorealistic datasets. Our experimental results demonstrate that our approach outperforms previous techniques in both quality and performance metrics for VR applications, enabling new possibilities for immersive photorealistic experiences.
- KSP Keywords
- Application-specific, Foveated rendering, Higher performance, Image regression, Novel approach, Real-Time applications, Style transfer, Visual Fidelity, immersive virtual reality, neural network(NN), neural network architecture