ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술지 Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
Cited 1 time in scopus Download 200 time Share share facebook twitter linkedin kakaostory
저자
장수영, 김형일
발행일
202208
출처
Sensors, v.22 no.15, pp.1-14
ISSN
1424-8220
출판사
MDPI
DOI
https://dx.doi.org/10.3390/s22155845
협약과제
22ZR1100, 자율적으로 연결·제어·진화하는 초연결 지능화 기술 연구, 박준희
초록
Effective exploration is one of the critical factors affecting performance in deep reinforcement learning. Agents acquire data to learn the optimal policy through exploration, and if it is not guaranteed, the data quality deteriorates, which leads to performance degradation. This study investigates the effect of initial entropy, which significantly influences exploration, especially in the early learning stage. The results of this study on tasks with discrete action space show that (1) low initial entropy increases the probability of learning failure, (2) the distributions of initial entropy for various tasks are biased towards low values that inhibit exploration, and (3) the initial entropy for discrete action space varies with both the initial weight and task, making it hard to control. We then devise a simple yet powerful learning strategy to deal with these limitations, namely, entropy-aware model initialization. The proposed algorithm aims to provide a model with high initial entropy to a deep reinforcement learning algorithm for effective exploration. Our experiments showed that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed.
KSP 제안 키워드
Action space, Critical factors, Data Quality, Deep reinforcement learning, Learning Speed, Learning Stage, Learning strategy, Model initialization, Optimal policy, Reinforcement Learning(RL), Reinforcement learning algorithm
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
저작자 표시 (CC BY)