상세정보
학술지
Learning to Discriminate Information for Online Action Detection: Analysis and Application
- 발행일
- 202305
- 출처
- IEEE Transactions on Pattern Analysis and Machine Intelligence, v.45 no.5, pp.5918-5934
- ISSN
- 0162-8828
- 출판사
- IEEE
- 협약과제
- 21HS4800, 장기 시각 메모리 네트워크 기반의 예지형 시각지능 핵심기술 개발, 문진영
- 초록
- Online action detection, which aims to identify an ongoing action from a streaming video, is an important subject in real-world applications. For this task, previous methods use recurrent neural networks for modeling temporal relations in an input sequence. However, these methods overlook the fact that the input image sequence includes not only the action of interest but background and irrelevant actions. This would induce recurrent units to accumulate unnecessary information for encoding features on the action of interest. To overcome this problem, we propose a novel recurrent unit, named Information Discrimination Unit (IDU), which explicitly discriminates the information relevancy between an ongoing action and others to decide whether to accumulate the input information. This enables learning more discriminative representations for identifying an ongoing action. In this paper, we further present a new recurrent unit, called Information Integration Unit (IIU), for action anticipation. Our IIU exploits the outputs from IDN as pseudo action labels as well as RGB frames to learn enriched features of observed actions effectively. In experiments on TVSeries and THUMOS-14, the proposed methods outperform state-of-the-art methods by a significant margin in online action detection and action anticipation. Moreover, we demonstrate the effectiveness of the proposed units by conducting comprehensive ablation studies.
- KSP 제안 키워드
- Image sequence, Input information, Online Action Detection, Real-world applications, Recurrent Neural Network(RNN), Streaming video, Temporal relations, information integration, state-of-The-Art