Fast Bootstrap Methodology for Regression Model Selection

Abstract

Using Resampling Methods Like Cross-Validation and Bootstrap is a Necessity in Neural Network Design, for Solving the Problem of Model Structure Selection. the Bootstrap is a Powerful Method Offering a Low Variance of the Model Generalization Error Estimate. Unfortunately, its Computational Load May Be Excessive When Used to Select among Neural Networks Models of Different Structures or Complexities. This Paper Presents the Fast Bootstrap (Fb) Methodology to Select the Best Model Structure; This Methodology is Applied Here to Regression Tasks. the Fast Bootstrap Assumes that the Computationally Expensive Term Estimated by the Bootstrap, the Optimism, is Usually a Smooth Function (Low-Order Polynomial) of the Complexity Parameter. Approximating the Optimism Term Makes It Possible to Considerably Reduce the Necessary Number of Simulations. the Fb Methodology is Illustrated on Multi-Layer Perceptrons, Radial-Basis Function Networks and Least-Square Support Vector Machines. © 2004 Published by Elsevier B.v.

Department(s)

Engineering Management and Systems Engineering

Comments

Academy of Finland, Grant 44886

Keywords and Phrases

Bootstrap; Model selection; Nonlinear modeling; Resampling

International Standard Serial Number (ISSN)

0925-2312

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Elsevier, All rights reserved.

Publication Date

01 Mar 2005

Link to Full Text

Share

 
COinS