BROWSE
Detail
Conference Paper
Anomaly Detection in Scientific Datasets using Sparse Representation
Cited 2 time in 
Share 
Share
- Authors
- Aekyeung Moon, Minjun Kim, Jiaxi Chen, Seung Woo Son
- Issue Date
- 2023-06
- Citation
- International Symposium on High-Performance Parallel and Distributed Computing (HPDC) 2023, pp.13-18
- Language
- English
- Type
- Conference Paper
- Abstract
- As the size and complexity of high-performance computing (HPC) systems keep growing, scientists' ability to trust the data produced is paramount due to potential data corruption for various reasons, which may stay undetected. While employing machine learning-based anomaly detection techniques could relieve scientists of such concern, it is practically infeasible due to the need for labels for volumes of scientific datasets and the unwanted extra overhead associated. In this paper, we exploit spatial sparsity profiles exhibited in scientific datasets and propose an approach to detect anomalies effectively. Our method first extracts block-level sparse representations of original datasets in the transformed domain. Then it learns from the extracted sparse representations and builds the boundary threshold between normal and abnormal without relying on labeled data. Experiments using real-world scientific datasets show that the proposed approach requires 13% on average (less than 10% in most cases and as low as 0.3%) of the entire dataset to achieve competitive detection accuracy (70.74%-100.0%) as compared to two state-of-the-art unsupervised techniques.
- KSP Keywords
- Detection accuracy, High Performance Computing, Labeled data, Learning-based, Real-world, Sparse representation, Spatial Sparsity, Unsupervised techniques, anomaly detection, data corruption, detection techniques