Learning High-Dimensional Parametric Maps Via Reduced Basis Adaptive Residual Networks
Abstract
We propose a scalable framework for the learning of high-dimensional parametric maps via adaptively constructed residual network (ResNet) maps between reduced bases of the inputs and outputs. When just few training data are available, it is beneficial to have a compact parametrization in order to ameliorate the ill-posedness of the neural network training problem. By linearly restricting high-dimensional maps to informed reduced bases of the inputs, one can compress high-dimensional maps in a constructive way that can be used to detect appropriate basis ranks, equipped with rigorous error estimates. A scalable neural network learning framework is thus to learn the nonlinear compressed reduced basis mapping. Unlike the reduced basis construction, however, neural network constructions are not guaranteed to reduce errors by adding representation power, making it difficult to achieve good practical performance. Inspired by recent approximation theory that connects ResNets to sequential minimizing flows, we present an adaptive ResNet construction algorithm. This algorithm allows for depth-wise enrichment of the neural network approximation, in a manner that can achieve good practical performance by first training a shallow network and then adapting. We prove universal approximation of the associated neural network class for Lν2 functions on compact sets. Our overall framework allows for constructive means to detect appropriate breadth and depth, and related compact parametrizations of neural networks, significantly reducing the need for architectural hyperparameter tuning. Numerical experiments for parametric PDE problems and a 3D CFD wing design optimization parametric map demonstrate that the proposed methodology can achieve remarkably high accuracy for limited training data, and outperformed other neural network strategies we compared against.
Recommended Citation
T. O'Leary-Roseberry et al., "Learning High-Dimensional Parametric Maps Via Reduced Basis Adaptive Residual Networks," Computer Methods in Applied Mechanics and Engineering, vol. 402, article no. 115730, Elsevier, Dec 2022.
The definitive version is available at https://doi.org/10.1016/j.cma.2022.115730
Department(s)
Mechanical and Aerospace Engineering
Keywords and Phrases
Adaptive surrogate construction; Control flows; Deep learning; Neural networks; Parametrized PDEs; Residual networks
International Standard Serial Number (ISSN)
0045-7825
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2023 Elsevier, All rights reserved.
Publication Date
01 Dec 2022
Comments
U.S. Department of Defense, Grant DE-AR0001208