BROWSE
Detail
Conference Paper
Towards Understanding Architectural Effects on Knowledge Distillation
Cited 0 time in 
Share 
Share
- Authors
- Ik-Hee Shin, Yong-Hyuk Moo, Yong-Ju Lee
- Issue Date
- 2020-10
- Citation
- International Conference on Information and Communication Technology Convergence (ICTC) 2020, pp.1144-1146
- Publisher
- IEEE
- Language
- English
- Type
- Conference Paper
- Abstract
- Knowledge distillation is a promising model compression solution, which adopts an apprenticeship (teacher-student) learning approach. Even referring to the same teacher network, it has reported different distillation performance according to a used student network architecture in current works. To tackle this issue, we investigate how scaling depth and width over layers influences learning capability based on distillation. Our experiment results show that depth scaling is a more determinant factor than width for selecting a smaller network in knowledge distillation.
- KSP Keywords
- Experiment results, Knowledge Distillation, Learning approach, Model compression, Network Architecture, learning capability