Direct Error Driven Learning for Classification in Applications Generating Big-Data
In this chapter, a comprehensive methodology is presented to address important data-driven challenges within the context of classification. First, it is demonstrated that challenges, such as heterogeneity and noise observed with big/large data-sets, affect the efficiency of a deep neural network (DNN)-based classifiers. To obviate these issues, a two-step classification framework is introduced where unwanted attributes (variables) are systematically removed through a preprocessing step and a DNN-based classifier is introduced to address heterogeneity in the learning process. Specifically, a multi-stage nonlinear dimensionality reduction (NDR) approach is described in this chapter to remove unwanted variables and a novel optimization framework is presented to address heterogeneity. In NDR, the dimensions are first divided into groups (grouping stage) and redundancies are then systematically removed in each group (transformation stage). The two-stage NDR procedure is repeated until a user-defined criterion controlling information loss is satisfied. The reduced dimensional data is finally used for classification with a DNN-based framework where direct error-driven learning regime is introduced. Within this framework, an approximation of generalization error is obtained by generating additional samples from the data. An overall error, which consists of learning and approximation of generalization error, is determined and utilized to derive a performance measure for each layer in the DNN. A novel layer-wise weight-tuning law is finally obtained through the gradient of this layer-wise performance measure where the overall error is directly utilized for learning. The efficiency of this two-step classification approach is demonstrated using various data-sets.
R. Krishnan et al., "Direct Error Driven Learning for Classification in Applications Generating Big-Data," Development and Analysis of Deep Learning Architectures, vol. 867, pp. 1 - 29, Springer Verlag, Nov 2020.
The definitive version is available at https://doi.org/10.1007/978-3-030-31764-5_1
Electrical and Computer Engineering
Mathematics and Statistics
Intelligent Systems Center
Second Research Center/Lab
Center for High Performance Computing Research
Keywords and Phrases
Big-Data; Deep Learning; Dimensionality Reduction; Learning Regime
International Standard Book Number (ISBN)
Book - Chapter
© 2020 Springer Nature Switzerland AG, All rights reserved.
01 Nov 2020