Title

Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter Optimization

Abstract

Today's extreme-scale high-performance computing (HPC) applications are producing volumes of data too large to save or transfer because of limited storage space and I/O bandwidth. Error-bounded lossy compression has been commonly known as one of the best solutions to the big science data issue, because it can significantly reduce the data volume with strictly controlled data distortion based on user requirements. In this work, we develop an adaptive parameter optimization algorithm integrated with a series of optimization strategies for SZ, a state-of-the-art prediction-based compression model. Our contribution is threefold. (1) We exploit effective strategies by using 2nd-order regression and 2nd-order Lorenzo predictors to improve the prediction accuracy significantly for SZ, thus substantially improving the overall compression quality. (2) We design an efficient approach selecting the best-fit parameter setting, by conducting a comprehensive priori compression quality analysis and exploiting an efficient online controlling mechanism. (3) We evaluate the compression quality and performance on a supercomputer with 4,096 cores, as compared with other state-of-the-art error-bounded lossy compressors. Experiments with multiple real-world HPC simulations datasets show that our solution can improve the compression ratio up to 46% compared with the second-best compressor. Moreover, the parallel I/O performance is improved by up to 40% thanks to the significant reduction of data size.

Meeting Name

29th International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2020 (2020: Jun. 23-26, Stockholm, Sweden)

Department(s)

Computer Science

Comments

This research was supported by the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC

Keywords and Phrases

High-Performance Computing; Lossy Compression; Parameter Optimization; Rate Distortion; Science Data

International Standard Book Number (ISBN)

978-145037052-3

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2020 Association for Computing Machinery (ACM), All rights reserved.

Publication Date

23 Jun 2020

Share

 
COinS