BROWSE
Detail
Journal Article
Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification
- Authors
- Doyeob Yeo, Min-Suk Kim, Ji-Hoon Bae
- Issue Date
- 2021-04
- Citation
- Applied Sciences, v.11, no.8, pp.1-16
- ISSN
- 2076-3417
- Publisher
- MDPI
- Language
- English
- Type
- Journal Article
- Abstract
- A deep-learning technology for knowledge transfer is necessary to advance and optimize efficient knowledge distillation. Here, we aim to develop a new adversarial optimization-based knowledge transfer method involved with a layer-wise dense flow that is distilled from a pre-trained deep neural network (DNN). Knowledge distillation transferred to another target DNN based on adversarial loss functions has multiple flow-based knowledge items that are densely extracted by overlapping them from a pre-trained DNN to enhance the existing knowledge. We propose a semisupervised learning-based knowledge transfer with multiple items of dense flow-based knowledge extracted from the pre-trained DNN. The proposed loss function would comprise a supervised cross-entropy loss for a typical classification, an adversarial training loss for the target DNN and discriminators, and Euclidean distance-based loss in terms of dense flow. For both pre-trained and target DNNs considered in this study, we adopt a residual network (ResNet) architecture. We propose methods of (1) the adversarial-based knowledge optimization, (2) the extended and flow-based knowledge transfer scheme, and (3) the combined layer-wise dense flow in an adversarial network. The results show that it provides higher accuracy performance in the improved target ResNet compared to the prior knowledge transfer methods.
- KSP Keywords
- Accuracy performance, Adversarial Training, Adversarial network, And Euclidean distance, Cross entropy, Deep neural network(DNN), Dense flow, Distance-based, Entropy loss, Flow-based(FB), Image Classification
This work is distributed under the term of Creative Commons License (CCL)
(CC BY)
(CC BY)
