Direct Error Driven Learning for Classification in Applications Generating Big-Data
Abstract
In this chapter, a comprehensive methodology is presented to address important data-driven challenges within the context of classification. First, it is demonstrated that challenges, such as heterogeneity and noise observed with big/large data-sets, affect the efficiency of a deep neural network (DNN)-based classifiers. To obviate these issues, a two-step classification framework is introduced where unwanted attributes (variables) are systematically removed through a preprocessing step and a DNN-based classifier is introduced to address heterogeneity in the learning process. Specifically, a multi-stage nonlinear dimensionality reduction (NDR) approach is described in this chapter to remove unwanted variables and a novel optimization framework is presented to address heterogeneity. In NDR, the dimensions are first divided into groups (grouping stage) and redundancies are then systematically removed in each group (transformation stage). The two-stage NDR procedure is repeated until a user-defined criterion controlling information loss is satisfied. The reduced dimensional data is finally used for classification with a DNN-based framework where direct error-driven learning regime is introduced. Within this framework, an approximation of generalization error is obtained by generating additional samples from the data. An overall error, which consists of learning and approximation of generalization error, is determined and utilized to derive a performance measure for each layer in the DNN. A novel layer-wise weight-tuning law is finally obtained through the gradient of this layer-wise performance measure where the overall error is directly utilized for learning. The efficiency of this two-step classification approach is demonstrated using various data-sets.
Recommended Citation
R. Krishnan et al., "Direct Error Driven Learning for Classification in Applications Generating Big-Data," Development and Analysis of Deep Learning Architectures, vol. 867, pp. 1 - 29, Springer Verlag, Nov 2020.
The definitive version is available at https://doi.org/10.1007/978-3-030-31764-5_1
Department(s)
Electrical and Computer Engineering
Second Department
Mathematics and Statistics
Research Center/Lab(s)
Intelligent Systems Center
Second Research Center/Lab
Center for High Performance Computing Research
Keywords and Phrases
Big-Data; Deep Learning; Dimensionality Reduction; Learning Regime
International Standard Book Number (ISBN)
978-3-030-31763-8
Document Type
Book - Chapter
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2020 Springer Nature Switzerland AG, All rights reserved.
Publication Date
01 Nov 2020
Comments
This research was supported in part by an NSF I/UCRC award IIP 1134721 and Intelligent Systems Center.
Part of the Studies in Computational Intelligence book series.