BROWSE
Detail
Conference Paper
Parameter Reduction For Deep Neural Network Based Acoustic Models Using Sparsity Regularized Factorization Neurons
Cited 4 time in 
Download 0 time
Share 
Download 0 time
Share
- Authors
- Hoon Chung, Euisok Chung, Jeon Gue Park, Ho-Young Jung
- Issue Date
- 2019-07
- Citation
- International Joint Conference on Neural Networks (IJCNN) 2019, pp.1-5
- Language
- English
- Type
- Conference Paper
- Project Code
- 19HS2500, Development of semi-supervised learning language intelligence technology and Korean tutoring service for foreigners, Lee Yunkeun
- Abstract
- In this paper, we propose a deep neural network (DNN) model parameter reduction technique for an efficient acoustic model. One of the most common DNN model parameter reduction techniques is to use low-rank matrix approximation. Although it can reduce a significant number of model parameters, there are two problems to be considered; one is the performance degradation, and the other is the appropriate rank selection. To solve these problems, retraining is carried out, and so-called explained variance is used. However, retraining takes additional time, and explained variance is not directly related to classification performance.Therefore, to mitigate these problems, we propose an approach that performs model parameter reduction simultaneously during model training from the aspect of minimizing classification error. The proposed method uses the product of three factorized matrices instead of a dense weight matrix, and applies sparsity constraint to make entries of the center diagonal matrix zero. After finishing training, a parameter-reduced model can be obtained by discarding the left and right vectors corresponding to zero entries within the center diagonal matrix.
- KSP Keywords
- Classification Performance, Deep neural network(DNN), Low-Rank Matrix Approximation, Model parameter, Reduced model, Reduction technique, acoustic model, classification error, diagonal matrix, parameter reduction, performance degradation