ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술대회 Towards Understanding Architectural Effects on Knowledge Distillation
Cited 0 time in scopus Download 9 time Share share facebook twitter linkedin kakaostory
저자
신익희, 문용혁, 이용주
발행일
202010
출처
International Conference on Information and Communication Technology Convergence (ICTC) 2020, pp.1144-1146
DOI
https://dx.doi.org/10.1109/ICTC49870.2020.9289630
협약과제
20HS3300, 부하분산과 능동적 적시 대응을 위한 빅데이터 엣지 분석 기술 개발, 이용주
초록
Knowledge distillation is a promising model compression solution, which adopts an apprenticeship (teacher-student) learning approach. Even referring to the same teacher network, it has reported different distillation performance according to a used student network architecture in current works. To tackle this issue, we investigate how scaling depth and width over layers influences learning capability based on distillation. Our experiment results show that depth scaling is a more determinant factor than width for selecting a smaller network in knowledge distillation.
KSP 제안 키워드
Experiment results, Learning Capability, Learning approach, Model compression, Network Architecture, knowledge distillation