ETRI-Knowledge Sharing Plaform

ENGLISH

성과물

논문 검색
구분 SCI
연도 ~ 키워드

상세정보

학술지 High-Performance Geospatial Big Data Processing System Based on MapReduce
Cited 20 time in scopus Download 27 time Share share facebook twitter linkedin kakaostory
저자
조정희, 이강우
발행일
201810
출처
ISPRS International Journal of Geo-Information, v.7 no.10, pp.1-18
ISSN
2220-9964
출판사
MDPI AG
DOI
https://dx.doi.org/10.3390/ijgi7100399
협약과제
18GH1200, 공간 빅데이터 저장관리 인프라 기술 개발 , 이강우
초록
With the rapid development of Internet of Things (IoT) technologies, the increasing volume and diversity of sources of geospatial big data have created challenges in storing, managing, and processing data. In addition to the general characteristics of big data, the unique properties of spatial data make the handling of geospatial big data even more complicated. To facilitate users implementing geospatial big data applications in a MapReduce framework, several big data processing systems have extended the original Hadoop to support spatial properties. Most of those platforms, however, have included spatial functionalities by embedding them as a form of plug-in. Although offering a convenient way to add new features to an existing system, the plug-in has several limitations. In particular, while executing spatial and nonspatial operations by alternating between the existing system and the plug-in, additional read and write overheads have to be added to the workflow, significantly reducing performance efficiency. To address this issue, we have developed Marmot, a high-performance, geospatial big data processing system based on MapReduce. Marmot extends Hadoop at a low level to support seamless integration between spatial and nonspatial operations of a solid framework, allowing improved performance of geoprocessing workflow. This paper explains the overall architecture and data model of Marmot as well as the main algorithm for automatic construction of MapReduce jobs from a given spatial analysis task. To illustrate how Marmot transforms a sequence of operators for spatial analysis to map and reduce functions in a way to achieve better performance, this paper presents an example of spatial analysis retrieving the number of subway stations per city in Korea. This paper also experimentally demonstrates that Marmot generally outperforms SpatialHadoop, one of the top plug-in based spatial big data frameworks, particularly in dealing with complex and time-intensive queries involving spatial index.
키워드
Big data, Geospatial applications, Geospatial big data, Hadoop, IoT, MapReduce
KSP 제안 키워드
Automatic construction, Big Data Processing, Big Data frameworks, Data Model, Data processing system, Geoprocessing workflow, Geospatial applications, Geospatial big data, High performance, Internet of thing(IoT), MapReduce framework
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
저작자 표시 (CC BY)