Distributed Min-Max Learning Scheme for Neural Networks with Applications to High-Dimensional Classification
In this article, a novel learning methodology is introduced for the problem of classification in the context of high-dimensional data. In particular, the challenges introduced by high-dimensional data sets are addressed by formulating a L1 regularized zero-sum game where optimal sparsity is estimated through a two-player game between the penalty coefficients/sparsity parameters and the deep neural network weights. In order to solve this game, a distributed learning methodology is proposed where additional variables are utilized to derive layerwise cost functions. Finally, an alternating minimization approach developed to solve the problem where the Nash solution provides optimal sparsity and compensation through the classifier. The proposed learning approach is implemented in a parallel and distributed environment through a novel computational algorithm. The efficiency of the approach is demonstrated both theoretically and empirically with nine data sets.
K. Raghavan et al., "Distributed Min-Max Learning Scheme for Neural Networks with Applications to High-Dimensional Classification," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 10, pp. 4323-4333, Institute of Electrical and Electronics Engineers (IEEE), Oct 2021.
The definitive version is available at https://doi.org/10.1109/TNNLS.2020.3017434
Mathematics and Statistics
Keywords and Phrases
Distributed Optimization; Machine Learning; Neural Networks
International Standard Serial Number (ISSN)
Article - Journal
© 2021 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
01 Oct 2021