BROWSE
Detail
Conference Paper
기계학습 학습률 제어를 위한 목적함수의 예측 곡률과 고속 Idle and Go 알고리즘
Cited - time in 
Share 
Share
- Issue Date
- 2020-11
- Citation
- 대한전자공학회 학술 대회 (추계) 2020, pp.608-611
- Publisher
- 대한전자공학회
- Language
- Korean
- Type
- Conference Paper
- Abstract
- In this paper, we propose a learning rate selection scheme without inner loops for search range optimization in stochastic gradient descent algorithms. Searching algorithms based on conventional nonlinear optimization techniques require an inner loop to find optimal learning rates. As the inner loops require additional computation, conventional nonlinear techniques for selection of learning rates are not suitable for learning huge-scale data set. If we disassemble the inner loops and the learning processes select optimal learning rates at each epoch or iteration, conventional nonlinear optimization techniques can be applied to machine learning. Moreover, in the procedure of learning rate selection unified to the learning process, we provide a fast selection scheme for learning rate employing estimation of the initial value in an attempt to minimize the needless additional computation. The proposed algorithm performed better in learning speed and classification compared to conventional learning schemes in numerical experiments.
- KSP Keywords
- Data sets, Gradient descent algorithm, Initial value, Inner loop, Learning Process, Learning Speed, Learning rate, Numerical experiments, Optimal learning, Optimization techniques, Rate selection