상세정보
학술대회
HEMVIP: Human Evaluation of Multiple Videos in Parallel
- 저자
- Patrik Jonell, 윤영우, Pieter Wolfert, Taras Kucherenko, Gustav Henter
- 발행일
- 202110
- 출처
- International Conference on Multimodal Interaction (ICMI) 2021, pp.707-711
- 협약과제
- 21HS1500, 고령 사회에 대응하기 위한 실환경 휴먼케어 로봇 기술 개발, 이재연
- 초록
- In many research areas, for example motion and gesture generation, objective measures alone do not provide an accurate impression of key stimulus traits such as perceived quality or appropriateness. The gold standard is instead to evaluate these aspects through user studies, especially subjective evaluations of video stimuli. Common evaluation paradigms either present individual stimuli to be scored on Likert-type scales, or ask users to compare and rate videos in a pairwise fashion. However, the time and resources required for such evaluations scale poorly as the number of conditions to be compared increases. Building on standards used for evaluating the quality of multimedia codecs, this paper instead introduces a framework for granular rating of multiple comparable videos in parallel. This methodology essentially analyses all condition pairs at once. Our contributions are 1) a proposed framework, called HEMVIP, for parallel and granular evaluation of multiple video stimuli and 2) a validation study confirming that results obtained using the tool are in close agreement with results of prior studies using conventional multiple pairwise comparisons.
- KSP 제안 키워드
- Gesture generation, Gold standard, Human evaluation, Perceived quality, User study, Validation study, Video stimuli, objective measure, pairwise comparisons, subjective evaluation
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
