Abstract
Nowadays, data reduction is becoming increasingly important in dealing with the large amounts of scientific data. Existing multilevel compression algorithms offer a promising way to manage scientific data at scale but may suffer from relatively low performance and reduction quality. In this paper, we propose MGARD+, a multilevel data reduction and refactoring framework drawing on previous multilevel methods, to achieve high-performance data decomposition and high-quality error-bounded lossy compression. Our contributions are four-fold: 1) We propose to leverage a level-wise coefficient quantization method, which uses different error tolerances to quantize the multilevel coefficients. 2) We propose an adaptive decomposition method which treats the multilevel decomposition as a preconditioner and terminates the decomposition process at an appropriate level. 3) We leverage a set of algorithmic optimization strategies to significantly improve the performance of multilevel decomposition/recompositing. 4) We evaluate our proposed method using four real-world scientific datasets and compare with several state-of-the-art lossy compressors. Experiments demonstrate that our optimizations improve the decomposition/recompositing performance of the existing multilevel method by up to $70 \times$70x, and the proposed compression method can improve compression ratio by up to $2 \times$2x compared with other state-of-the-art error-bounded lossy compressors under the same level of data distortion.
Recommended Citation
X. Liang and B. Whitney and J. Chen and L. Wan and Q. Liu and D. Tao and J. Kress and D. Pugmire and M. Wolf and N. Podhorszki and S. Klasky, "MGARD+: Optimizing Multilevel Methods for Error-Bounded Scientific Data Reduction," IEEE Transactions on Computers, vol. 71, no. 7, pp. 1522 - 1536, Institute of Electrical and Electronics Engineers, Jul 2022.
The definitive version is available at https://doi.org/10.1109/TC.2021.3092201
Department(s)
Computer Science
Keywords and Phrases
error control; High-performance computing; lossy compression; multilevel decomposition; scientific data
International Standard Serial Number (ISSN)
1557-9956; 0018-9340
Document Type
Article - Journal
Document Version
Final Version
File Type
text
Language(s)
English
Rights
© 2023 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
01 Jul 2022
