BROWSE
Detail
Conference Paper
Evaluating an LLM-Based Semantic Parser in a Restricted Mobile Manipulation Simulation
Cited - time in 
Share 
Share
- Authors
- Jinseo Lee, Hyunwoo Joe
- Issue Date
- 2026-08
- Citation
- 한국인공지능학회 학술 대회 (하계) 2026, pp.1-5
- Publisher
- 한국인공지능학회
- Language
- English
- Type
- Conference Paper
- Abstract
- Natural language instructions must be converted into structured task representations before they can be executed by a robot controller. This paper evaluates an LLM-based semantic parser in a restricted mobile manipulation simulation. The parser maps English instructions to a JavaScript Object Notation (JSON)-formatted representation with predefined object and destination labels, and a deterministic Webots Supervisor controller executes the parsed task in simulation. The implementation uses Webots ground-truth state information and kinematic object attachment; therefore, this paper evaluates instruction parsing behavior rather than real-world perception, obstacle-aware planning, or force-based manipulation. Two instruction datasets are used for evaluation: a prompt-aligned dataset and a held-out dataset containing unseen paraphrases, ambiguous expressions, and out-of-domain references. The parser achieves 98.0% exact-match accuracy (EM) on the prompt-aligned dataset and 74.0% EM on the held-out dataset, with average inference latencies of 558.10 ms and 558.75 ms, respectively. These results indicate that the parser can produce correct object-destination task labels for explicit instructions in the restricted simulation, while semantic paraphrases and ambiguous instructions remain challenging.
- Keyword
- Large Language Model, Semantic Parsing, Natural Language Instruction, Mobile Manipulation
- KSP Keywords
- Force-based, Javascript Object Notation(JSON), Language Models, Mobile Manipulation, Natural language, Real-world, Robot controller, Semantic parser, Semantic parsing, State information, ground truth