BROWSE
Detail
Conference Paper
Retrieve-and-generate: Image-to-3D framework with mesh retrieval and texture generation
Cited 0 time in 
Share 
Share
- Authors
- SangEun Lee, Wonseok Chae, Hyeon-Jin Kim
- Issue Date
- 2024-10
- Citation
- International Conference on Information and Communication Technology Convergence (ICTC) 2024, pp.881-885
- Publisher
- IEEE
- Language
- English
- Type
- Conference Paper
- Abstract
- In recent years, significant advancements have been made in the field of image-to-3D reconstruction. However, one of the persistent challenges that remains unresolved is messy surfaces of generated outputs. To tackle this issue, we introduce a novel retrieve-and-generate scheme specifically designed for image-to-3D reconstruction tasks. Our method involves a two-stage process: first, the model retrieves the most similar 3D mesh corresponding to the input image; second, it generates a texture map that can be accurately mapped onto the retrieved mesh. In the initial retrieval phase, our approach leverages a pre-trained multi-modal joint representation to identify the 3D mesh that closely resembles the input image within the embedding space. Subsequently, texture generation module generates a realistic texture reflecting the input image, which leads to the complete 3D object reconstruction when this texture is mapped to the retrieved mesh. We have observed that our retrieve-and-generate approach significantly enhances the quality of the reconstructed 3D objects from a single input image. This improvement in reconstruction performance demonstrates the efficacy of our proposed method and its potential to advance the state-of-the-art in image-to-3D reconstruction technology.
- KSP Keywords
- 3D framework, 3D mesh, 3D object reconstruction, 3D reconstruction technology, Embedding space, Joint representation, Multi-modal, Reconstruction performance, Single-input, Texture generation, Texture map