상세정보
학술지
Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
- 발행일
- 202208
- 출처
- Sensors, v.22 no.15, pp.1-14
- ISSN
- 1424-8220
- 출판사
- MDPI
- 협약과제
- 22ZR1100, 자율적으로 연결·제어·진화하는 초연결 지능화 기술 연구, 박준희
- 초록
- Effective exploration is one of the critical factors affecting performance in deep reinforcement learning. Agents acquire data to learn the optimal policy through exploration, and if it is not guaranteed, the data quality deteriorates, which leads to performance degradation. This study investigates the effect of initial entropy, which significantly influences exploration, especially in the early learning stage. The results of this study on tasks with discrete action space show that (1) low initial entropy increases the probability of learning failure, (2) the distributions of initial entropy for various tasks are biased towards low values that inhibit exploration, and (3) the initial entropy for discrete action space varies with both the initial weight and task, making it hard to control. We then devise a simple yet powerful learning strategy to deal with these limitations, namely, entropy-aware model initialization. The proposed algorithm aims to provide a model with high initial entropy to a deep reinforcement learning algorithm for effective exploration. Our experiments showed that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed.
- KSP 제안 키워드
- Action space, Critical factors, Data Quality, Deep reinforcement learning, Learning Speed, Learning Stage, Learning strategy, Model initialization, Optimal policy, Reinforcement Learning(RL), Reinforcement learning algorithm
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
