Residual Variance Estimation in Machine Learning
Abstract
The Problem of Residual Variance Estimation Consists of Estimating the Best Possible Generalization Error Obtainable by Any Model based on a Finite Sample of Data. Even Though It is a Natural Generalization of Linear Correlation, Residual Variance Estimation in its General Form Has Attracted Relatively Little Attention in Machine Learning. in This Paper, We Examine Four Different Residual Variance Estimators and Analyze their Properties Both Theoretically and Experimentally to Understand Better their Applicability in Machine Learning Problems. the Theoretical Treatment Differs from Previous Work by Being based on a General Formulation of the Problem Covering Also Heteroscedastic Noise in Contrary to Previous Work, Which Concentrates on Homoscedastic and Additive Noise. in the Second Part of the Paper, We Demonstrate Practical Applications in Input and Model Structure Selection. the Experimental Results Show that using Residual Variance Estimators in These Tasks Gives Good Results Often with a Reduced Computational Complexity, While the Nearest Neighbor Estimators Are Simple and Easy to Implement. © 2009 Elsevier B.v. All Rights Reserved.
Recommended Citation
E. Liitiäinen et al., "Residual Variance Estimation in Machine Learning," Neurocomputing, vol. 72, no. 16 thru 18, pp. 3692 - 3703, Elsevier, Oct 2009.
The definitive version is available at https://doi.org/10.1016/j.neucom.2009.07.004
Department(s)
Engineering Management and Systems Engineering
Keywords and Phrases
Input selection; Model structure selection; Nearest neighbor; Noise variance estimation; Nonparametric estimator; Residual variance
International Standard Serial Number (ISSN)
0925-2312
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2024 Elsevier, All rights reserved.
Publication Date
01 Oct 2009
