ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술대회 Rank Selection of CP-decomposed Convolutional Layers with Variational Bayesian Matrix Factorization
Cited 7 time in scopus Download 0 time Share share facebook twitter linkedin kakaostory
저자
마셀라, 이승익, 서범수
발행일
201802
출처
International Conference on Advanced Communications Technology (ICACT) 2018, pp.347-350
DOI
https://dx.doi.org/10.23919/ICACT.2018.8323750
협약과제
18HR1500, 안전한 무인이동체를 위한 ICT 기반기술 개발, 안재영
초록
Convolutional Neural Networks (CNNs) is one of successful method in many areas such as image classification tasks. However, the amount of memory and computational cost needed for CNNs inference obstructs them to run efficiently in mobile devices because of memory and computational ability limitation. One of the method to compress CNNs is compressing the layers iteratively, i.e. by layer-by-layer compression and fine-tuning, with CP-decomposition in convolutional layers. To compress with CP-decomposition, rank selection is important. In the previous approach rank selection that is based on sensitivity of each layer, the average rank of the network was still arbitrarily selected. Additionally, the rank of all layers were decided before whole process of iterative compression, while the rank of a layer can be changed after fine-tuning. Therefore, this paper proposes selecting rank of each layer using Variational Bayesian Matrix Factorization (VBMF) which is more systematic than arbitrary approach. Furthermore, to consider the change of each layer's rank after fine-tuning of previous iteration, the method is applied just before compressing the target layer, i.e. after fine-tuning of the previous iteration. The results show better accuracy while also having more compression rate in AlexNet's convolutional layers compression.
KSP 제안 키워드
Compression rate, Convolution neural network(CNN), Image classification, Iterative compression, Layer-by-Layer(LbL), Matrix Factorization, Mobile devices, Whole process, computational cost, fine-tuning, rank selection