BROWSE
Detail
Conference Paper
Characterization of Transform-Based Lossy Compression for HPC Datasets
Cited 1 time in 
Share 
Share
- Authors
- Aekyeung Moon, Jiaxi Chen, Seung Woo Son, Minjun Kim
- Issue Date
- 2022-11
- Citation
- International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD) 2022, pp.56-62
- Language
- English
- Type
- Conference Paper
- Abstract
- As the scale and complexity of high-performance computing (HPC) systems keep growing, data compression techniques are often adopted to reduce the data volume and processing time. While lossy compression becomes preferable to a lossless one because of the potential benefit of generating a high compression ratio, it would lose its worth the effort without finding an optimal balance between volume reduction and information loss. Among many lossy compression techniques, transform-based lossy algorithms utilize spatial redundancy bet-ter. However, the transform-based lossy compressor has received relatively less attention because there is a lack of understanding of its compression performance on scientific data sets. The insight of this paper is that, in transform-based lossy compressors, quantifying dominant coefficients at the block level reveals the right balance, potentially impacting overall compression ratios. Motivated by this, we characterize three transformation-based lossy compression mechanisms with different information com-paction methods using the statistical features that capture data characteristics. And then, we build several prediction models using the statistical features and the characteristics of dominant coefficients and evaluate the effectiveness of each model using six HPC datasets from three production-level simulations at scale. Our results demonstrate that the random forest classifier captures the behavior of dominant coefficients precisely, achieving nearly 99 % of prediction accuracy.
- KSP Keywords
- Compression performance, Compression techniques, Data Volume, Data characteristics, Data sets, High-performance computing(HPC), Information loss, Prediction accuracy, Random Forest Classifier, Scientific data, Spatial redundancy