BROWSE
Detail
Conference Paper
Accelerating Training of DNN in Distributed Machine Learning System with Shared Memory
Cited 7 time in 
Download 2 time
Share 
Download 2 time
Share
- Authors
- Eun-Ji Lim, Shin-Young Ahn, Wan Choi
- Issue Date
- 2017-10
- Citation
- International Conference on Information and Communication Technology Convergence (ICTC) 2017, pp.1210-1213
- Language
- English
- Type
- Conference Paper
- Project Code
- 17HS1900, Development of HPC System for Accelerating Large-scale Deep Learning, Choi Wan
- Abstract
- In distributed DNN training, the speed of reading and updating model parameters greatly affects model training time. In this paper we investigate the performance of deep neural network training with parameter sharing based on shared memory for distributed machine learning. We propose a shared memory-based modification of the deep learning framework. In our framework, remote shared memory is used to maintain global shared parameters of parallel deep learning workers. Our framework can accelerate training of DNN by speeding up the parameter sharing in every training iteration in distributed model training. We evaluated our proposed framework by training the three different deep learning model. The experiment results show that our framework improves training time for deep learning models in distributed system.
- KSP Keywords
- Deep learning framework, Deep neural network(DNN), Distributed System(DS), Distributed machine learning, Experiment results, Machine learning system, Memory-based, Model parameter, Neural network training, Shared Memory, Shared parameters