Maintaining Trust in Reduction: Preserving the Accuracy of Quantities of Interest for Lossy Compression
Abstract
As the growth of data sizes continues to outpace computational resources, there is a pressing need for data reduction techniques that can significantly reduce the amount of data and quantify the error incurred in compression. Compressing scientific data presents many challenges for reduction techniques since it is often on non-uniform or unstructured meshes, is from a high-dimensional space, and has many Quantities of Interests (QoIs) that need to be preserved. To illustrate these challenges, we focus on data from a large scale fusion code, XGC. XGC uses a Particle-In-Cell (PIC) technique which generates hundreds of PetaBytes (PBs) of data a day, from thousands of timesteps. XGC uses an unstructured mesh, and needs to compute many QoIs from the raw data, f.
One critical aspect of the reduction is that we need to ensure that QoIs derived from the data (density, temperature, flux surface averaged momentums, etc.) maintain a relative high accuracy. We show that by compressing XGC data on the high-dimensional, nonuniform grid on which the data is defined, and adaptively quantizing the decomposed coefficients based on the characteristics of the QoIs, the compression ratios at various error tolerances obtained using a multilevel compressor (MGARD) increases more than ten times. We then present how to mathematically guarantee that the accuracy of the QoIs computed from the reduced f is preserved during the compression. We show that the error in the XGC density can be kept under a user-specified tolerance over 1000 timesteps of simulation using the mathematical QoI error control theory of MGARD, whereas traditional error control on the data to be reduced does not guarantee the accuracy of the QoIs.
Recommended Citation
Q. Gong et al., "Maintaining Trust in Reduction: Preserving the Accuracy of Quantities of Interest for Lossy Compression," Communications in Computer and Information Science, vol. 1512, pp. 22 - 39, Springer, Mar 2022.
The definitive version is available at https://doi.org/10.1007/978-3-030-96498-6_2
Meeting Name
Smoky Mountains Computational Sciences and Engineering Conference, SMC 2021 (2021: Oct. 18-20, Virtual)
Department(s)
Computer Science
Keywords and Phrases
Error Control; Lossy Compression; Quantities of Interest; XGC Simulation Data
International Standard Book Number (ISBN)
978-303096497-9
International Standard Serial Number (ISSN)
1865-0937; 1865-0929
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2022 Springer, All rights reserved.
Publication Date
10 Mar 2022
Comments
This research was supported by the ECP CODAR, Sirius-2, and RAPIDS-2 projects through the Advanced Scientific Computing Research (ASCR) program of Department of Energy, and the LDRD project through DRD program of Oak Ridge National Laboratory.