Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter Optimization
Abstract
Today's extreme-scale high-performance computing (HPC) applications are producing volumes of data too large to save or transfer because of limited storage space and I/O bandwidth. Error-bounded lossy compression has been commonly known as one of the best solutions to the big science data issue, because it can significantly reduce the data volume with strictly controlled data distortion based on user requirements. In this work, we develop an adaptive parameter optimization algorithm integrated with a series of optimization strategies for SZ, a state-of-the-art prediction-based compression model. Our contribution is threefold. (1) We exploit effective strategies by using 2nd-order regression and 2nd-order Lorenzo predictors to improve the prediction accuracy significantly for SZ, thus substantially improving the overall compression quality. (2) We design an efficient approach selecting the best-fit parameter setting, by conducting a comprehensive priori compression quality analysis and exploiting an efficient online controlling mechanism. (3) We evaluate the compression quality and performance on a supercomputer with 4,096 cores, as compared with other state-of-the-art error-bounded lossy compressors. Experiments with multiple real-world HPC simulations datasets show that our solution can improve the compression ratio up to 46% compared with the second-best compressor. Moreover, the parallel I/O performance is improved by up to 40% thanks to the significant reduction of data size.
Recommended Citation
K. Zhao et al., "Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter Optimization," Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (2020, Stockholm, Sweden), pp. 89 - 100, Association for Computing Machinery (ACM), Jun 2020.
The definitive version is available at https://doi.org/10.1145/3369583.3392688
Meeting Name
29th International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2020 (2020: Jun. 23-26, Stockholm, Sweden)
Department(s)
Computer Science
Keywords and Phrases
High-Performance Computing; Lossy Compression; Parameter Optimization; Rate Distortion; Science Data
International Standard Book Number (ISBN)
978-145037052-3
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2020 Association for Computing Machinery (ACM), All rights reserved.
Publication Date
23 Jun 2020
Comments
This research was supported by the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC