Multi-Party Sparse Discriminant Learning

Abstract

Sparse Discriminant Analysis (SDA) has been widely used to improve the performance of classical Fisher's Linear Discriminant Analysis in supervised metric learning, feature selection and classification. With the increasing needs of distributed data collection, storage and processing, enabling the Sparse Discriminant Learning to embrace the Multi-Party distributed computing environments becomes an emerging research topic. This paper proposes a novel Multi-Party SDA algorithm, which can learn SDA models effectively without sharing any raw data and basic statistics among machines. The proposed algorithm 1) leverages the direct estimation of SDA [1] to derive a distributed loss function for the discriminant learning, 2) parameterizes the distributed loss function with local/global estimates through bootstrapping, and 3) approximates a global estimation of linear discriminant projection vector by optimizing the "distributed bootstrapping loss function" with gossip-based stochastic gradient descent. Experimental results on both synthetic and real-world benchmark datasets show that our algorithm can compete with the centralized SDA with similar performance, and significantly outperforms the most recent distributed SDA [2] in terms of accuracy and F1-score.

Meeting Name

17th IEEE International Conference on Data Mining, ICDM (2017: Nov. 18-21, New Orleans, LA)

Department(s)

Computer Science

Second Department

Mathematics and Statistics

Keywords and Phrases

Multi-Party Statistical Learning; Sparse Discriminant Analysis; Bayesian Asymptotic Efficiency

International Standard Book Number (ISBN)

978-1-5386-3835-4

International Standard Serial Number (ISSN)

2374-8486

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

21 Nov 2017

Link to Full Text

Share

 
COinS