ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술지 Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification
Cited 2 time in scopus Download 297 time Share share facebook twitter linkedin kakaostory
저자
여도엽, 김민석, 배지훈
발행일
202104
출처
Applied Sciences, v.11 no.8, pp.1-16
ISSN
2076-3417
출판사
MDPI
DOI
https://dx.doi.org/10.3390/app11083720
초록
A deep-learning technology for knowledge transfer is necessary to advance and optimize efficient knowledge distillation. Here, we aim to develop a new adversarial optimization-based knowledge transfer method involved with a layer-wise dense flow that is distilled from a pre-trained deep neural network (DNN). Knowledge distillation transferred to another target DNN based on adversarial loss functions has multiple flow-based knowledge items that are densely extracted by overlapping them from a pre-trained DNN to enhance the existing knowledge. We propose a semisupervised learning-based knowledge transfer with multiple items of dense flow-based knowledge extracted from the pre-trained DNN. The proposed loss function would comprise a supervised cross-entropy loss for a typical classification, an adversarial training loss for the target DNN and discriminators, and Euclidean distance-based loss in terms of dense flow. For both pre-trained and target DNNs considered in this study, we adopt a residual network (ResNet) architecture. We propose methods of (1) the adversarial-based knowledge optimization, (2) the extended and flow-based knowledge transfer scheme, and (3) the combined layer-wise dense flow in an adversarial network. The results show that it provides higher accuracy performance in the improved target ResNet compared to the prior knowledge transfer methods.
KSP 제안 키워드
Accuracy performance, Adversarial Training, Adversarial network, And Euclidean distance, Cross-Entropy, Deep neural network(DNN), Dense flow, Distance-based, Entropy loss, Flow-based, Image classification
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
저작자 표시 (CC BY)