ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술지 Adaptive Metadata Rebalance in Exascale File System
Cited 6 time in scopus Download 8 time Share share facebook twitter linkedin kakaostory
저자
차명훈, 김동오, 김홍연, 김영균
발행일
201704
출처
Journal of Supercomputing, v.73 no.4, pp.1337-1359
ISSN
0920-8542
출판사
Springer
DOI
https://dx.doi.org/10.1007/s11227-016-1812-x
협약과제
15MS5400, (ICBMS-총괄) ICBMS(IoT, 클라우드, 빅데이터, 모바일, 정보보호) 핵심 기술 개발 사업 총괄 및 엑사스케일급 클라우드 스토리지 기술 개발, 김영균
초록
This paper presents an effective method of metadata rebalance in exascale distributed file systems. Exponential data growth has led to the need for an adaptive and robust distributed file system whose typical architecture is composed of a large cluster of metadata servers and data servers. Though each metadata server can have an equally divided subset from the entire metadata set at first, there will eventually be a global imbalance in the placement of metadata among metadata servers, and this imbalance worsens over time. To ensure that disproportionate metadata placement will not have a negative effect on the intrinsic performance of a metadata server cluster, it is necessary to recover the balanced performance of the cluster periodically. However, this cannot be easily done because rebalancing seriously hampers the normal operation of a file system. This situation continues to get worse with both an ever-present heavy workload on the file system and frequent failures of server components at exascale. As one of the primary reasons for such a degraded performance, file system clients frequently fail to look up metadata from the metadata server cluster during the period of metadata rebalance; thus, metadata operations cannot proceed at their normal speed. We propose a metadata rebalance model that minimizes failures of metadata operations during the metadata rebalance period and validate the proposed model through a cost analysis. The analysis results demonstrate that our model supports the feasibility of online metadata rebalance without the normal operation obstruction and increases the chances of maintaining balance in a huge cluster of metadata servers.
키워드
Distributed file system, Exascale, Metadata, Rebalance
KSP 제안 키워드
Balanced performance, Distributed File system, Large cluster, Metadata server, Negative effects, Over time, Proposed model, Server cluster, cost analysis, need for, normal operation