BROWSE
Detail
Conference Paper
Contrastive learning based multimodal object recognition using ambiguous natural language
Cited 0 time in 
Share 
Share
- Authors
- Joonyoung Jung, Dong-oh Kang
- Issue Date
- 2024-10
- Citation
- International Conference on Control, Automation and Systems (ICCAS) 2024, pp.51-54
- Publisher
- IEEE
- Language
- English
- Type
- Conference Paper
- Abstract
- To enhance communication between humans and robots, it is essential to improve the ability of robots to comprehend the meaning of ambiguous natural language (ANL) used by humans. Artificial intelligence models have insufficient training with ANL. Therefore, this paper proposes a contrastive learning (CL) based multimodal object recognition system using ANL. In the proposed system, the model is trained to have high similarity for positive data, where there is an association between ANL and object, and low similarity for negative data, where there is no such association. After training, an evaluation test on multimodal object retrieval was conducted to select object related to ANL sentence from multiple objects. Also, this evaluation test examined the performance variation according to the size of the multimodal training data. In the multimodal object retrieval experiment, the top-1 and top-2 retrieval accuracies were 0.7 and 0.87, respectively, despite using a small amount of training data.
- KSP Keywords
- Amount of training data, Artificial intelligence models, Multiple objects, Natural language, Negative data, Performance Variation, Positive data, object recognition system, object retrieval