ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술지 Learning to Discriminate Information for Online Action Detection: Analysis and Application
Cited 2 time in scopus Download 10 time Share share facebook twitter linkedin kakaostory
저자
이수민, 은현준, 문진영, 최석연, 김윤형, 정찬호, 김창익
발행일
202305
출처
IEEE Transactions on Pattern Analysis and Machine Intelligence, v.45 no.5, pp.5918-5934
ISSN
0162-8828
출판사
IEEE
DOI
https://dx.doi.org/10.1109/TPAMI.2022.3204808
협약과제
21HS4800, 장기 시각 메모리 네트워크 기반의 예지형 시각지능 핵심기술 개발, 문진영
초록
Online action detection, which aims to identify an ongoing action from a streaming video, is an important subject in real-world applications. For this task, previous methods use recurrent neural networks for modeling temporal relations in an input sequence. However, these methods overlook the fact that the input image sequence includes not only the action of interest but background and irrelevant actions. This would induce recurrent units to accumulate unnecessary information for encoding features on the action of interest. To overcome this problem, we propose a novel recurrent unit, named Information Discrimination Unit (IDU), which explicitly discriminates the information relevancy between an ongoing action and others to decide whether to accumulate the input information. This enables learning more discriminative representations for identifying an ongoing action. In this paper, we further present a new recurrent unit, called Information Integration Unit (IIU), for action anticipation. Our IIU exploits the outputs from IDN as pseudo action labels as well as RGB frames to learn enriched features of observed actions effectively. In experiments on TVSeries and THUMOS-14, the proposed methods outperform state-of-the-art methods by a significant margin in online action detection and action anticipation. Moreover, we demonstrate the effectiveness of the proposed units by conducting comprehensive ablation studies.
KSP 제안 키워드
Image sequence, Input information, Online Action Detection, Real-world applications, Recurrent Neural Network(RNN), Streaming video, Temporal relations, information integration, state-of-The-Art