ETRI-Knowledge Sharing Plaform

KOREAN
논문 검색
Type SCI
Year ~ Keyword

Detail

Conference Paper Recognition of Voice and Hand Activities Through Fusion of Acceleration and Speech
Cited - time in scopus Download 1 time Share share facebook twitter linkedin kakaostory
Authors
Young-Giu Jung, Chang Seok Bae, Mun-Sung Han
Issue Date
2008-12
Citation
International Conference on Semantic and Digital Media Technologies (SAMT) 2008, pp.67-74
Language
English
Type
Conference Paper
Project Code
08MH1600, Research on Human-friendly Next Generation PC Technology Standardization on u-Computing, Han Mun Sung
Abstract
Hand activity and speech comprise the most important modalities of human-to-agent interaction. So a multimodal interface can achieve more natural and effective human-agent interaction. In this paper, we suggest a novel technique for improving the performance of accelerometer-based hand activity recognition system using fusion of speech. The speech data is used in our experiment as the complementary sensor data to the acceleration data in an attempt to improve the performance of hand activity recognizer. This recognizer is designed to be capable of classifying nineteen hand activities. It consists of 10 natural gestures, e.g., ‘go left’, ‘over here’ and 9 emotional expressions by hand activity, e.g., ‘I feel hot’, ‘I love you’. To improve performance of hand activity recognition using feature fusion, we propose a modified Time Delay Neural Network (TDNN) architecture with a dedicated fusion layer and a time normalization layer. Our experimental result shows that the performance of this system yields an improvement of about 6.96% compared to the use of accelerometers alone.
KSP Keywords
Acceleration data, Activity Recognition, Emotional expression, Experimental Result, Feature Fusion, Fusion layer, Human-Agent Interaction, Multimodal interface, Novel technique, Recognition System, Time delay neural network