Abstract
The increase of computer processing speed is significantly outpacing improvements in network and storage bandwidth, leading to the big data challenge in modern science, where scientific applications can quickly generate much more data than that can be transferred and stored. As a result, big scientific data must be reduced by a few orders of magnitude while the accuracy of the reduced data needs to be guaranteed for further scientific explorations. Moreover, scientists are often interested in some specific spatial/temporal regions in their data, where higher accuracy is required. The locations of the regions requiring high accuracy can sometimes be prescribed based on application knowledge, while other times they must be estimated based on general spatial/temporal variation. In this paper, we develop a novel multilevel approach which allows users to impose region-wise compression error bounds. Our method utilizes the byproduct of a multilevel compressor to detect regions where details are rich and we provide the theoretical underpinning for region-wise error control. With spatially varying precision preservation, our approach can achieve significantly higher compression ratios than single-error bounded compression approaches and control errors in the regions of interest. We conduct the evaluations on two climate use cases-one targeting small-scale, node features and the other focusing on long, areal features. For both use cases, the locations of the features were unknown ahead of the compression. By selecting approximately 16% of the data based on multi-scale spatial variations and compressing those regions with smaller error tolerances than the rest, our approach improves the accuracy of post-analysis by approximately 2 x compared to single-error-bounded compression at the same compression ratio. Using the same error bound for the region of interest, our approach can achieve an increase of more than 50% in overall compression ratio.
Recommended Citation
Q. Gong and B. Whitney and C. Zhang and X. Liang and A. Rangarajan and J. Chen and L. Wan and P. Ullrich and Q. Liu and R. Jacob and S. Ranka and S. Klasky, "Region-Adaptive, Error-Controlled Scientific Data Compression using Multilevel Decomposition," ACM International Conference Proceeding Series, article no. 5, Association for Computing Machinery (ACM), Jul 2022.
The definitive version is available at https://doi.org/10.1145/3538712.3538717
Department(s)
Computer Science
Keywords and Phrases
Climate Data Compression; Error Control; Region-adaptive Lossy Compression
International Standard Book Number (ISBN)
978-145039667-7
Document Type
Article - Conference proceedings
Document Version
Final Version
File Type
text
Language(s)
English
Rights
© 2023 Association for Computing Machinery, All rights reserved.
Publication Date
06 Jul 2022
Comments
U.S. Department of Energy, Grant None