상세정보
학술지
Prefetching-Based Metadata Management in Advanced Multitenant Hadoop
- 발행일
- 201902
- 출처
- Journal of Supercomputing, v.75 no.2, pp.533-553
- ISSN
- 0920-8542
- 출판사
- Springer
- 협약과제
- 16MH1700, (통합)스마트 네트워킹 핵심 기술 개발, 양선희
- 초록
- Metadata management is an essential part in Apache Hadoop. Performing optimization of metadata accesses enhances big data storing, processing and analyzing, especially in multitenant environments. Nevertheless, as environmental complexity increases, metadata management is becoming more challenging and costly because of the heavy performance issues. In this paper, we propose a novel approach to improve the performance of metadata management for Hadoop in the multitenant environment based on the prefetching mechanism. We create metadata access graphs based on historical access values, define access patterns and then perform prefetching potential items for the near-future requests to minimize the latency. We present a formal algorithm to apply the prefetching mechanism into the Hadoop system and perform the actual implementation on a recent Hadoop system. Experimental results show that the proposed approach can enable the high performance for metadata management as well as maintain advanced multitenancy features.
- KSP 제안 키워드
- Access graphs, Access pattern, Apache Hadoop, Big Data, Hadoop system, High performance, Novel approach, metadata management, multitenant environment