Learning High-Dimensional Parametric Maps Via Reduced Basis Adaptive Residual Networks
We propose a scalable framework for the learning of high-dimensional parametric maps via adaptively constructed residual network (ResNet) maps between reduced bases of the inputs and outputs. When just few training data are available, it is beneficial to have a compact parametrization in order to ameliorate the ill-posedness of the neural network training problem. By linearly restricting high-dimensional maps to informed reduced bases of the inputs, one can compress high-dimensional maps in a constructive way that can be used to detect appropriate basis ranks, equipped with rigorous error estimates. A scalable neural network learning framework is thus to learn the nonlinear compressed reduced basis mapping. Unlike the reduced basis construction, however, neural network constructions are not guaranteed to reduce errors by adding representation power, making it difficult to achieve good practical performance. Inspired by recent approximation theory that connects ResNets to sequential minimizing flows, we present an adaptive ResNet construction algorithm. This algorithm allows for depth-wise enrichment of the neural network approximation, in a manner that can achieve good practical performance by first training a shallow network and then adapting. We prove universal approximation of the associated neural network class for Lν2 functions on compact sets. Our overall framework allows for constructive means to detect appropriate breadth and depth, and related compact parametrizations of neural networks, significantly reducing the need for architectural hyperparameter tuning. Numerical experiments for parametric PDE problems and a 3D CFD wing design optimization parametric map demonstrate that the proposed methodology can achieve remarkably high accuracy for limited training data, and outperformed other neural network strategies we compared against.
T. O'Leary-Roseberry et al., "Learning High-Dimensional Parametric Maps Via Reduced Basis Adaptive Residual Networks," Computer Methods in Applied Mechanics and Engineering, vol. 402, article no. 115730, Elsevier, Dec 2022.
The definitive version is available at https://doi.org/10.1016/j.cma.2022.115730
Mechanical and Aerospace Engineering
Keywords and Phrases
Adaptive surrogate construction; Control flows; Deep learning; Neural networks; Parametrized PDEs; Residual networks
International Standard Serial Number (ISSN)
Article - Journal
© 2023 Elsevier, All rights reserved.
01 Dec 2022
U.S. Department of Defense, Grant DE-AR0001208